Learnijoy
ResearchAI ResearchAI Engineering & DevTools

Emergent Sparsity in Frozen CNNs for Reinforcement Learning.

Scott M. Norton· July 31, 2026 View original

Key takeaways

  • Deep reinforcement learning agents exhibit emergent sparsity in fully-connected layers when using frozen random CNN feature extractors.
  • This sparsity scales with task complexity and is critical for agent performance.
  • The information bottleneck forms early in training, long before significant reward accumulation.
  • Understanding emergent sparsity can lead to more efficient and interpretable DRL architectures.

Who benefits

AI ResearchGamingRoboticsAutonomous SystemsOptimization

Summary

This research reveals that deep reinforcement learning agents, when using frozen, randomly initialized CNN feature extractors, spontaneously develop extremely sparse fully-connected representations without explicit sparsity objectives. This emergent sparsity scales with task complexity and is crucial for performance.

This paper reports a surprising phenomenon in deep reinforcement learning (DRL): when DRL agents are trained with frozen, randomly initialized Convolutional Neural Network (CNN) feature extractors, their subsequent fully-connected layers spontaneously develop highly sparse representations. This sparsity emerges without any explicit objective to induce it. For instance, in deterministic Pong, agents compress task-relevant information into as few as 1-3 neurons out of 64, significantly fewer than trainable CNNs. The study establishes four key findings: first, the degree of sparsity correlates with task complexity; second, within-game scaling shows that the usable dimensionality of the random projection can limit achievable performance; third, ablating these active neurons crashes performance, confirming their necessity; and fourth, the information bottleneck forms early in training, long before positive rewards are observed. This research suggests that gradient descent on a frozen random projection can reveal the effective rank of the underlying problem, offering insights into how DRL agents learn and represent information, even without explicit sparsity mechanisms.

Why it matters

Professionals in AI research and engineering can leverage this understanding of emergent sparsity to design more efficient DRL architectures, potentially reducing computational costs and improving interpretability without explicit sparsity-inducing techniques.

How to implement this in your domain

  1. 1Explore using frozen, randomly initialized feature extractors in DRL experiments to observe emergent sparsity and its impact on learning.
  2. 2Investigate the relationship between task complexity and the degree of sparsity in your DRL models to optimize network architecture.
  3. 3Consider the implications of early information bottleneck formation for training schedules and hyperparameter tuning in DRL.
  4. 4Apply insights from emergent sparsity to develop more computationally efficient and interpretable reinforcement learning agents.

Original post by Scott M. Norton

"arXiv:2607.26059v1 Announce Type: cross Abstract: We report a striking phenomenon: deep reinforcement learning agents trained with frozen, randomly initialized CNN feature extractors spontaneously develop extremely sparse fully-connected representations, without any sparsity-indu…"

View on X

Originally posted by Scott M. Norton on X · view source

Want to go deeper?

Turn these trends into skills with Learnijoy's hands-on AI & tech courses.

Explore courses

More in AI Research

AI ResearchAI Engineering & DevTools

New Framework Improves Partial Multi-View Clustering Performance.

DAS-PMVC is a novel framework for partial multi-view clustering that addresses view asymmetry and irrelevant samples by leveraging dual alignment and structure enhancement. It uses anchor graph structure alignment, structure-enhanced feature learning, and a dual alignment strategy to achieve superior clustering performance on various datasets.

Shubin Ma, Liang Zhao, Chuanye He, Zhenjiao Liu, Liang Zou, Lin Yuanbo Wu, Yu ShaoJul 31, 2026
AI Engineering & DevToolsAI Research

Dual Teachers Improve Adversarial Robustness and Accuracy.

This work extends Information Bottleneck Distillation (IBD) by introducing a "clean teacher" alongside a robust teacher to improve the robustness/accuracy tradeoff against adversarial attacks. The proposed method transfers features from both teachers to a student model, achieving better clean accuracy while maintaining adversarial robustness, outperforming original IBD and competing with state-of-the-art approaches.

Vincent Ryusuke Takahashi, Yoshinari Takeishi, Jun'ichi Takeuchi, Kave SalamatianJul 31, 2026
AI Engineering & DevToolsAI Research

Dynamic Batch Sizes Improve Large Language Model Training Efficiency.

This paper proposes a new approach to deep learning dynamics, deriving joint scaling laws for loss based on both learning rate and batch size schedules. It introduces an optimal dynamic batch size schedule that consistently outperforms static batch size baselines, highlighting its importance for large language model training.

Jiaxiang Li, Zhiqi Bu, Shiyun XuJul 31, 2026