Complex State Propagator Achieves 100% Accuracy in State Tracking

Transformer architectures, with their attention mechanisms, have become dominant in sequence modeling. However, for specific deterministic state tracking tasks, such as checking parity, modular counting, or matching parentheses, attention might be an unnecessary complexity. This paper argues that simple state propagation can be sufficient for these tasks. The proposed architecture, the Complex State Propagator (CSP), is a minimalistic recurrent model. It propagates complex-valued hidden states across layers, updating them via input-dependent rotations in the complex domain, without relying on output projections at intermediate steps. To ensure deep propagation without issues like gradient vanishing, CSP incorporates block-level skip connections, element-wise complex normalization, and SiLU activation at sequence boundaries. When combined with Focal Loss, CSP achieves perfect F1 scores and 100% accuracy across several canonical deterministic state tracking tasks. This demonstrates that for certain types of problems, a highly efficient, state-propagation-only model can outperform or match the performance of more complex attention-based models, suggesting a path towards simpler and more efficient architectures for specific applications.