Inference-Time Alignment Improves Fairness in Reinforcement Learning.

Deep reinforcement learning (RL) agents often struggle to adapt to new performance criteria like fairness once deployed, typically requiring costly retraining. This paper explores a novel approach inspired by large language model alignment, focusing on steering pretrained RL policies towards fairness objectives during inference. The proposed multiplicative policy shaping framework adjusts action probabilities based on action-dependent welfare scores, effectively modifying behavior without altering the underlying policy parameters. Extensive experiments across various domains demonstrate that this inference-time policy shaping significantly enhances welfare-based fairness while maintaining core task performance. This method offers a flexible and efficient way to incorporate new preferences into existing RL systems, addressing a key challenge in deploying adaptable and ethical AI.