Latent Self-Distillation Boosts Agent Learning Efficiency

A central challenge in the development of self-evolving AI is enabling agents to effectively learn from their experiences and integrate that knowledge into their operational policies. On-policy self-distillation (OPSD) has emerged as a promising approach, where a "self-teacher" provides supervision on the student agent's own trajectories. However, current OPSD methods often rely on pre-defined, human-engineered privileged artifacts such as specific answers, feedback, or skills, which limits their scalability and end-to-end learnability for continuous self-improvement. This research introduces Latent On-Policy Self-Distillation (LOPD), a significant advancement that addresses these limitations. Instead of prescribing a fixed form of privileged context, LOPD makes the teacher's privileged context itself learnable directly from experience. Technically, LOPD retrieves relevant past experiences and synthesizes them into continuous latent tokens. These tokens then condition the self-teacher, which provides dense, token-level supervision to the student agent as it generates trajectories based on the task and interaction history. The method also incorporates a privileged-margin objective to ensure stable and regulated learning of this latent context. Empirical evaluations demonstrate LOPD's superior performance, outperforming other OPSD methods and reinforcement learning baselines in tasks such as agentic tool use and code generation. Furthermore, LOPD exhibits high learning efficiency, achieving strong results with significantly less rollout budget than comparable methods. Ablation studies confirm that the ability to learn the privileged context is crucial for these observed gains, positioning LOPD as a step towards more autonomous and scalable agent evolution.