New Framework for Uncertainty-Aware Governance in Delegated AI Systems

As AI systems increasingly rely on delegating tasks to various specialized models, evaluators, and tools, the primary challenge shifts from mere model accuracy to effective uncertainty-aware governance. This involves determining the optimal level of autonomy for AI, calibrating trust based on evidence, understanding performance limits, and identifying when human intervention becomes critical. To address these challenges, a new framework called the Minimum Sufficient Oversight Principle (MSO) has been introduced. MSO is a variational principle designed to minimize the governance burden while ensuring delivery constraints are met. This approach leads to a "water-filling" allocation of governed delegation across different tasks, optimizing the balance between AI autonomy and human oversight. The framework includes a capacity theorem for review policies, a first-order approximation for workflow complexity and quality degradation, and a scaling law for intervention timing. It also highlights "masking" as a governance pathology where corrected performance can obscure the true competence signal. Practical prescriptions include upstream-first correction and sensitivity-based intervention, with a companion Python package available for implementation.