Cautious Context Steering Improves Language Model Personalization.

Personalizing large language models (LMs) to individual user preferences is crucial but often hindered by data sparsity and poor generalization in existing methods like per-user adapters or reward models. In-context learning (ICL) and Context Steering (CoS) offer alternatives by conditioning the base LM directly on user context, but they lack dynamic control over context influence or incur high inference costs. This research proposes Cautious Context Steering (CCS), which adds a small adapter to a frozen LM. This adapter learns to decide, at each token generation step, whether and how strongly the user context should influence the output. Trained on a single dataset, CCS demonstrates improved generation quality both in-domain and across various out-of-distribution personalization benchmarks, showcasing strong generalization. Crucially, CCS avoids the need for per-user fine-tuning and the double forward pass required by CoS, leading to substantial reductions in inference cost.