Self-CTRL Enhances LLM Transparency and Safety Through Consistent Explanations

Language models that can accurately describe their own operational logic and behavior are inherently more transparent, easier to audit, and build greater user trust. The challenge lies in ensuring these self-explanations faithfully reflect the model's actions. Researchers have developed Self-Consistency Training with Reinforcement Learning (Self-CTRL), a novel approach designed to optimize the alignment between a language model's internal explanations and its external behavior on related inputs. This method iteratively refines either the explanations to better predict behavior or the behavior to better match the explanations. Applied to tasks like probabilistic reasoning and constitutional AI, Self-CTRL significantly improved the correlation between self-reported and measured biases, and enhanced the fidelity of rules describing model refusals. It also reduced harmful behavior in constitutional AI settings without increasing refusals on harmless prompts, demonstrating a generalizable strategy for safer, more transparent, and controllable AI.