MASkills Optimizes Multi-Agent LLM Systems with Continual Learning

Multi-agent systems built upon Large Language Models (LLMs) have demonstrated impressive performance on intricate tasks. However, enabling these systems to continually improve from their interaction experiences remains a significant challenge. While existing self-reflection methods create experience memories, these memories are often difficult to invoke, refine, or scale effectively. Researchers propose that agent skills, defined as structured procedural knowledge specifying when and how to act and which resources to use, offer a more actionable unit for improvement. Introducing MASkills, a novel continual learning framework designed to optimize multi-agent LLM systems through the evolution of these agent skills. MASkills presents a new agent-optimization pipeline that integrates several key components: skill-conditioned credit assignment, hierarchical credit aggregation, and momentum-smoothed optimization. This integrated approach allows agent skill libraries to dynamically evolve through processes of refinement, induction of new skills, consolidation of existing ones, and pruning of ineffective skills. Experiments conducted on benchmarks such as HotpotQA, LoCoMo, and GAIA across various agentic tasks confirm the effectiveness of MASkills. The framework consistently demonstrates improved performance, showcasing its ability to enable multi-agent LLM systems to learn and adapt continually from their interactions.