KV-Skill Enables Modular, Efficient Knowledge Injection for LLMs.

Current methods for imparting task knowledge to language models either embed it in prompts, which require re-interpretation, or update model weights, making skills hard to manage. KV-Skill offers a novel alternative: external, factorized operators that a frozen LLM accesses via a lightweight interface, without adding prompt positions. KV-Skill supports two pathways: "Registration" converts authored text skills into text-derived operators, and "Reward learning" develops compact latent operators directly from task outcomes. Both methods significantly enhance the effectiveness of procedural knowledge. For instance, on LiveMath, KV-Skill registration achieved 77.2% accuracy, far surpassing source text skills and other methods. The framework also demonstrates superior performance against soft prefixes, prefix tuning, and LoRA under matched reward training and parameter budgets. Crucially, KV-Skill allows multiple independently loadable skills to be retained through a single shared interface without measurable forgetting, proving its modularity and efficiency for managing diverse expertise.