Ghost Attractor Networks Offer Efficient, Stable Robotic Action Decoding

Generating sequential outputs with large-scale Transformer and diffusion decoders often incurs high memory costs that scale with sequence length, alongside iterative per-step computation. While smaller feed-forward decoders offer efficiency, they typically produce unstructured latent representations, hindering closed-loop control features like phase-conditioned action generation and cross-step latent carry-over, which require stable latent basins. This paper introduces Ghost Attractor Networks, a theoretically derived dynamical decoder designed to address these limitations. Its latent state evolves under a learned potential with drift, inherently creating a basin-attractor structure. This design is motivated by the need for multi-modality, single-pass switching at the decoder level, and constant memory usage. Mode transitions in Ghost arise from saddle-node bifurcations with ghost-attractor escape, and a hierarchical phase-space decomposition separates initial basin convergence from subsequent proprioceptive refinement. Empirical evaluations show that a Ghost network, trained end-to-end, exhibits the predicted gradient-flow contraction. As a robotic action decoder, a 2.3-million-parameter Ghost matched the offline accuracy of a 1.07-billion-parameter Diffusion Transformer, achieving this with 462 times fewer parameters and 32 times lower latency. It also outperformed five other 2M-parameter decoders in offline mean squared error. In closed-loop benchmarks, phase conditioning on Ghost's basin-structured latent improved success rates significantly, with persistent-latent ensembling reaching a 95.7% success rate.