SESA Agents Evolve Skills Through Self-Play and Memory

Self-play agents are effective at generating training problems, but their learning curricula often lack persistent state, meaning failures don't explicitly shape future practice. Conversely, external skill memories preserve procedural experience but are typically learned from fixed task distributions, limiting their adaptability. The SESA (Self-Evolving Skill-Augmented Agent) framework bridges this gap by integrating procedural memory as an evolving state within tool-augmented search self-play. In SESA, a challenger agent poses problems, while a separate solver agent retrieves skills from memory. Informative failures encountered by the solver are then distilled into reusable skills and written back to this memory. This creates a bidirectional loop: the updated memory alters the solver's behavior and success rates, which in turn influences the challenger's reward and the distribution of future problems. This co-evolution of task generation and skill memory allows for continuous improvement. The benefits of retrieved skills can be incorporated into the model parameters for memory-free deployment or retained in the external bank for optional inference-time retrieval, demonstrating significant accuracy improvements on various question-answering benchmarks.