SkillSight Improves LLM Agent Skill Retrieval Accuracy and Speed

As Large Language Model (LLM) agents access increasingly vast libraries of skills, accurately retrieving the most relevant skill becomes a critical challenge. Current retrieval methods often treat skill descriptions like generic documents, failing to account for the common, non-discriminatory language that frequently appears across many skills. This shared descriptive background can inflate relevance scores and mask the unique signals that truly differentiate one skill from another. Researchers have developed SkillSight, a novel, training-free framework designed to overcome this bias. SkillSight employs two calibration techniques: Semantic Background Calibration, which estimates and reduces similarity caused by generic tokens, and Lexical Evidence Calibration, which downweights shared background tokens to highlight discriminative features. Experiments on SRA-Bench and SkillBench-Supp demonstrate that SkillSight consistently improves retrieval metrics, boosting Recall@10 by over 20 percentage points in some cases. It also significantly outperforms LLM Selection in end-to-end evaluations and is orders of magnitude faster than traditional dense retrieval methods, making skill selection both more accurate and efficient.