Spectral Distillation Learns Nonlinear Dynamics with Linear Models.

Learning and representing complex nonlinear dynamical systems is a significant challenge in many fields. This research presents "Spectral Distillation," a provable two-step pipeline that allows for the learning of such systems through a compact linear state-space representation, bypassing the need to directly solve non-convex system-identification problems. The process begins by observing an unknown nonlinear dynamical system. In the first step, an implicit spectral predictor is learned using Observation Spectral Filtering (OSF), a convex method known for its effectiveness. This OSF predictor is designed to compete with the best possible linear observer for the given system. The second step involves "spectral-to-LDS distillation," which converts the implicit OSF predictor into an explicit recurrent linear dynamical system (LDS). A key theoretical contribution is a main theorem demonstrating that the average prediction error of the distilled LDS decomposes into an exponentially small distillation term and an OSF learning term, which is governed by the Luenberger complexity of the best observer. This guarantee is notably dimension-free, depending on observer complexity rather than the latent dimension of the nonlinear system. This method represents the first end-to-end provable approach for extracting a best-in-hindsight LDS representation of nonlinear dynamics through convex learning and provable distillation. Experiments on linear LDS benchmarks and MuJoCo behavior cloning tasks show that this train-then-distill pipeline yields compact LDS predictors that match or surpass directly trained baselines.