RATTL Enhances Safe AI Decision-Making Under Uncertainty

AI agents operating in uncertain environments need to balance exploration with safety, especially when their knowledge of the environment is incomplete. This paper introduces RATTL (Risk-Adversarial Total-Reward Learning), a novel framework designed to quantify and manage risk in sequential decision-making. RATTL links an agent's level of caution directly to its epistemic uncertainty, which is its uncertainty about the unknown dynamics of the environment. The agent maintains a Bayesian posterior over these dynamics and plans against an ambiguity set whose radius shrinks as more evidence is gathered. This allows the agent's behavior to smoothly transition from worst-case robustness to risk-neutral optimization as its knowledge improves. The framework ensures that the agent's value lies between an uninformed robust value and the full-knowledge optimum, with the gap diminishing as uncertainty resolves. A practical example demonstrates how an agent might defer an efficient but risky action until it has sufficient confidence, highlighting RATTL's potential for enhancing runtime safety in AI systems, including LLM-based agents.