BlockTrain Enables Decentralized AI Training and Inference

The current landscape of frontier AI training heavily favors large organizations with access to centralized, high-density accelerator clusters, creating a significant barrier for open or independent AI initiatives. This concentration of resources makes independent AI development dependent on scarce capital, privileged infrastructure, and specific data center locations. To address this, researchers have developed Spheroid BlockTrain, a novel decentralized protocol for both AI training and inference. BlockTrain operates by partitioning a large model into smaller, independently trainable blocks. Each block is optimized by a local worker based on an objective derived from the global target, and these blocks are then assembled for inference. Experiments demonstrate that BlockTrain can achieve performance comparable to end-to-end Transformer references, with individual workers only needing to train a single block and avoiding the overhead of full-model optimizer state. The protocol also shows efficiency in inference, outperforming plain autoregressive pipelines by emitting full sequences per WAN traversal, making large-scale AI more accessible and less reliant on centralized infrastructure.