LARA Adapters Enable Efficient, Composable LLM Adaptation and Alignment

Adapting large language models (LLMs) for specific tasks or preferences typically involves fine-tuning, often using methods like LoRA (Low-Rank Adaptation) which modify the model's weights. This research introduces LARA (Lightweight Additive Residual Adaptation), an alternative approach that achieves efficient adaptation without altering the base model's weights. Instead of modifying weight matrices, LARA adds low-rank corrections directly to the residual stream of a frozen model at a select few layers. This method has been shown to match the performance of LoRA on tasks like code fine-tuning and preference optimization (DPO) while using a comparable number of parameters. A significant advantage of LARA is its ability to expose a scaling factor, gamma, during inference. This factor allows for smooth interpolation between the base model's behavior and the adapted behavior, offering a level of graded control not available with weight-space adaptation. Furthermore, because each adapted behavior is a small residual module over a shared frozen base, multiple behaviors can be held resident simultaneously and dynamically routed per token. This enables hosting numerous distinct behaviors on a single model with minimal overhead, making it highly efficient for scenarios requiring diverse, on-demand adaptations.