FRESCO: Frequency Domain Reservoir Computing for Efficient Recurrent Models.

The quadratic scaling bottleneck of transformers with sequence length has renewed interest in recurrent neural networks. Echo State Networks (ESNs) are a promising recurrent model type, known for their efficient training due to fixed recurrent weights, which avoids the need for backpropagation through time. However, ESNs typically require large "reservoirs" to achieve high expressivity, leading to an O(N^2) state-update bottleneck that limits their scalability compared to modern recurrent architectures. To overcome this limitation, researchers developed Frequency Domain Reservoir Computing (FRESCO). This novel ESN architecture operates entirely within the frequency domain, cleverly avoiding domain-shift overheads. FRESCO achieves an impressive O(N) complexity for dense, non-linear recurrent updates by employing a unique dimensional zero-padding input embedding, a packed frequency-domain readout, and a natively applied frequency-domain non-linearity. This design significantly reduces both computational costs and energy consumption during training and inference. Furthermore, FRESCO demonstrates state-of-the-art predictive performance across memory benchmarks, sequential classification, and multivariate long-horizon forecasting tasks, positioning it as a scalable and efficient alternative for dense recurrent models.