Distilled Skills Outperform Deep Reasoning in LLM Agents

Large language models (LLMs) operating in reasoning modes often achieve superior performance on multi-step agentic tasks compared to their non-reasoning counterparts. However, this comes at a significant cost, typically incurring a 3-6x premium in output tokens per episode, much of which is spent re-deriving procedures common across similar tasks. This paper introduces a method to amortize this recurring cost by distilling "skills." A coding agent analyzes a small corpus of existing task trajectories and compiles a compact natural-language skill. This skill is then injected into the system prompt of a non-reasoning model. The results are striking: across four agentic benchmarks (ALFWorld, tau^2-bench telecom and retail, and SpreadsheetBench-Verified), these distilled skills recovered 55% to over 100% of the reasoning gap for GPT-5.4-mini on held-out tasks. Crucially, the models using distilled skills often exceeded the performance of the reasoning mode outright on two benchmarks, while emitting 2.7-6x fewer output tokens and zero reasoning tokens. The research also found that reasoning traces are not a prerequisite for skill distillation; skills derived from non-reasoning trajectories alone remained competitive. This suggests that "wide search" across episodes to distill common procedures can be more efficient and effective than "deep search" (test-time reasoning) within each individual episode.