TRACE Improves Enterprise LLM Tool Retrieval with Business Rules.

Parametric retrieval allows Large Language Models (LLMs) to implicitly retrieve tools by assigning unique virtual tokens to APIs and training the model to generate them. However, this approach often leads to a loss of parametric tool knowledge during training and slow beam-search decoding, hindering real-time deployment. Researchers introduce TRACE (Tool Retrieval via Augmented Chain-of-thought and Enterprise rules), a two-stage curriculum designed to overcome these limitations. The first stage uses multi-format memorization supervised fine-tuning (SFT) to initialize tool knowledge with LoRA. The core innovation lies in Stage 2, where the model is trained to emit a "thinking trace" before generating a JSON list of tool tokens. This training incorporates both existing data and queries synthesized from domain-expert-curated business rules, all augmented with reasoning traces. This method not only preserves but improves tool understanding, achieving significant gains in MCQ and QA probing accuracy, and enables single-beam greedy decoding for production-level latency. TRACE demonstrated ~86% recall on one domain and ~60% on another, significantly outperforming embedding baselines.