Lightweight Random Attention Improves Mobile Sleep Staging Efficiency.

Mobile sleep staging is crucial for in-home sleep monitoring, but current sequential models like RNNs and Transformers are often too computationally intensive for deployment on mobile or wearable devices. This limitation hinders the widespread adoption of real-time, closed-loop sleep modulation systems. To address this, a new paper introduces Random Attention (RA), a lightweight temporal modeling module. RA utilizes fixed random projections to replace complex learnable sequence modeling with a more efficient similarity-based aggregation approach. This design adds minimal parameters beyond the epoch encoder, yet effectively enables temporal smoothing. The researchers provide a theoretical interpretation of RA through the Random Attention Prior Kernel (RAPK), which clarifies its function as a combination of global smoothing and feature similarity. Experiments on Sleep-EDF-20 and Sleep-EDF-78 datasets show that RA consistently boosts accuracy and F1 score by 1-3% over epoch-wise baselines, achieving competitive performance with more resource-intensive models like LSTM, GRU, and Transformer, while demonstrating strong generalization and robustness.