YB Mixer: New Integrable Token Mixing Layer for Stable Sequence Processing

A new sequence token mixing layer, dubbed the YB Mixer, has been developed based on principles from integrable systems, specifically free fermion and generalized Yang Baxter structures. This innovative design applies a core concept where a local algebraic constraint guarantees global computational stability within the model. By utilizing the Ising exchange algebra, the mixer creates a free fermionic structure that functions as an exactly norm-preserving orthogonal map. This inherent orthogonality contributes significantly to the model's stability and prevents issues like vanishing or exploding gradients during processing. Furthermore, the algebra produces commuting transfer matrices, which enable order-free inference and adaptability to varying computational budgets. The YB Mixer also incorporates a spectral circulant generator to generalize to longer sequence lengths while maintaining its crucial orthogonal and commuting properties, offering a highly stable and mathematically grounded architecture for sequence processing.