Framework Quantifies Systemic AI Harms in Critical Infrastructure

A new research paper introduces a comprehensive framework designed to quantify the system-level harms that can arise from integrating AI into complex sociotechnical systems, particularly Critical National Infrastructure (CNI). The authors argue that current AI evaluation methods are too model-centric and fail to provide insight into how AI failures translate into broader systemic risks involving technical, human, and organizational elements. The proposed framework bridges this gap by connecting structured hazard analysis, component-level testing, and probabilistic system modeling. This integrated approach creates a traceable pathway from specific AI model behaviors to their potential system-wide consequences, allowing practitioners to understand the "so what?" of AI failures and quantify their overall impact. This enables a more evidence-based and anticipatory approach to AI governance in critical contexts. As an illustrative example, the framework was applied to the UK's Real Time Gross Settlement (RTGS) financial system. Researchers derived AI-driven loss scenarios using Systems Theoretic Process Analysis (STPA) and examined adversarial manipulation of LLM-based trading. Component-level experiments showed that even simple adversarial inputs could induce behavioral shifts in AI recommendations, which, when mapped to a financial contagion model, increased bank failures and lowered the threshold for cascading disruption, especially with widespread AI adoption.