SkillBoost Mitigates LLM Agent Overfitting in Self-Evolution

A significant challenge in deploying large language model (LLM) agents in real-world applications is their ability to learn and reuse experiences from past interactions. Treating skills as trainable states, similar to neural network parameters, offers a promising solution. However, optimizing these skills based on limited real-environment trajectories often leads to overfitting, where agents become too specialized to the training data. Overexploiting these trajectories can cause overfitting to the current batch, while unconstrained exploration risks regressing on previously mastered tasks. To address this tension between exploration and exploitation, researchers propose SkillBoost, a three-stage framework designed to mitigate skill overfitting. SkillBoost employs structured exploitation to pinpoint failures in editable skill components, prior-guided exploration that leverages the LLM's existing knowledge to generate diverse repair candidates, and a verified acceptance stage that only commits a candidate if it improves performance within a defined regression bound. Extensive experiments across 23 model-benchmark configurations demonstrate that SkillBoost achieves state-of-the-art performance while effectively preventing overfitting, surpassing both human-crafted and LLM-generated skills. Furthermore, optimized skills show transferability, allowing other agents to reuse them on similar tasks.