SKILLER Boosts Small LLM Agent Performance with Language-Level RL

Researchers have developed SKILLER, a novel reinforcement learning framework designed to enhance the capabilities of Small Language Models (SLMs) by automatically generating reusable, executor-specific skills. While agent skills are crucial for constraining LLM behavior and ensuring high-quality task execution, deploying them with powerful closed-source models often incurs prohibitive inference costs. The rise of open-source SLMs deployable on consumer-grade GPUs presents an opportunity to reduce these costs, but generating effective skills tailored for these compact models remains a challenge. SKILLER addresses this by employing a strong, larger model to act as both the actor and critic, while treating the small-model agent system as the environment. All reinforcement learning signals are propagated entirely through natural language, allowing the framework to generate skills that are highly optimized for the target SLM. This language-level approach enables SLMs to learn and execute complex tasks more effectively. Extensive evaluations across five benchmarks using Qwen3.5-9B and Qwen3.5-4B demonstrated that SKILLER significantly outperforms existing skill generation methods. It achieved absolute gains of 4.3 to 20.4 percentage points for the 9B model and 1.8 to 13.3 points for the 4B model, remarkably matching the performance of strong closed-source models on single-skill tasks. This breakthrough makes advanced agent capabilities more accessible and cost-effective for real-world deployment.