Simple Logit Bias Adapts LLMs Without Fine-Tuning.

Adapting large language models (LLMs) for specific internal tasks often involves complex fine-tuning or ad-hoc prompt engineering, which can be operationally challenging and raise privacy concerns with sensitive data. A simpler, more accessible method for adaptation is highly desirable. This paper explores a minimal alternative: adapting LLMs by simply biasing their logits with a user-defined vector at the API level. Researchers developed a black-box method to learn a single, context-independent logit-bias vector that is added at every decoding step, without requiring modifications to model weights or gradients. Starting from a KL-regularized reinforcement learning objective, the study characterized when such a fixed logit-bias vector can approximate optimal prefix-dependent corrections and derived a closed-form inverse-propensity estimator. Empirically, this straightforward decoding-time intervention significantly improved base models on mathematical and reasoning benchmarks. Crucially, it achieved these gains using substantially fewer trainable parameters compared to conventional fine-tuning, presenting a lightweight and efficient mechanism for adapting language models with minimal access requirements.