Risk-Aware Causal Gating Enhances LLM Safety in High-Stakes Decisions

A novel framework called Risk-Aware Causal Gating (RACG) has been introduced to enhance the safety and reliability of modern decision systems that increasingly rely on learned components. These AI models can sometimes produce confident but incorrect outputs, leading to costly errors in downstream actions. RACG addresses this by making informed decisions on whether to execute a model's prediction, defer it, or abstain entirely. The core of RACG involves modeling the causal pathway from potential actions to their outcomes. Instead of relying solely on raw predictive confidence, it gates each decision based on an estimated counterfactual risk. To ensure reliability, the framework provides distribution-free bounds on the probability of acting under high-risk conditions, translating these into operating thresholds that meet user-specified safety constraints. Furthermore, RACG incorporates an adaptive gating policy that can adjust to shifts in data distribution. It achieves this by continuously monitoring discrepancies between predicted and actual outcomes, tightening the gate when causal assumptions appear to be violated. Evaluations across simulated interventions and real-world benchmarks demonstrated that RACG substantially reduces high-cost errors while largely preserving the utility of an ungated policy, outperforming traditional confidence-based and selective-prediction baselines.