Quantum-Inspired Substrate Accelerates Sequence Model Training.

Sequence models, whether based on attention or recurrence, typically rely on a real-valued hidden state and an affine-softmax readout for predictions. This research explores a fundamental alternative: a complex-valued substrate inspired by quantum theory, where information is encoded in the phases of the state and scores are derived from quadratic Born forms. Previous theoretical work suggested this complex substrate could be representationally more powerful than real models with linear readouts. This study investigates whether it also offers faster training. By relaxing strict unitarity and the Born vocabulary readout for practical deployment, the researchers instantiated this complex-valued substrate within both the Mamba state-space model and an attention-based Transformer. When trained on byte-level corpora, the complex versions of these models, matched in parameter count, consistently reached target validation losses in significantly fewer optimization steps. Specifically, the state-space model achieved this in approximately one-third and the attention model in about one-half of the steps compared to their real-valued counterparts. The performance advantage of the state-space model continued to widen after the learning-rate warmup phase, suggesting a sustained benefit beyond initial training. For the attention model, the advantage was primarily an early-training effect, decaying towards zero on all corpora. This "quantum shortcut" demonstrates that the choice of numerical substrate can profoundly impact training efficiency, offering a novel direction for accelerating the development of sequence models.