Hard-Routed MoR-LoRA Composes Frozen Reasoning Experts Efficiently.

Composing independently trained LoRA (Low-Rank Adaptation) adapters into a single large language model (LLM) is a valuable technique for adapting models to multiple domains, especially when the original training data cannot be shared. A common approach involves Mixture-of-Experts (MoE) style routing, but soft weighted combinations can alter the unit-scale additive update under which each LoRA module was originally trained. This research introduces Hard-Routed MoR-LoRA, a two-stage framework designed for composing frozen reasoning LoRA experts through unit-scale hard selection. In the first stage, domain-specific LoRA adapters are independently trained using reinforcement learning from verifiable feedback to create specialized reasoning experts. In the second stage, all experts are frozen, and their reasoning traces are distilled. Only a lightweight shared router and a small attention LoRA are then trained for integration. The router employs hard top-1 routing, selecting exactly one expert per token, with a straight-through estimator enabling gradient-based training. Experiments across various benchmarks and model scales demonstrate that Hard-Routed MoR-LoRA effectively preserves expert behavior while requiring significantly fewer trainable parameters compared to soft-routing mixture baselines. Analysis also suggests that soft mixtures often concentrate routing mass on a single expert, reinforcing the efficiency of hard unit-scale routing for frozen LoRA expert composition.