LLM Guardrails Face Verdict Staleness in Adaptive Systems

This paper addresses a significant vulnerability in self-adaptive systems (SAS) that rely on large language model (LLM) guardrails: "verdict staleness." This occurs when an LLM's approval, correct at the time of checking, becomes invalid by the time it is actually used, creating a time-of-check to time-of-use (TOCTOU) hazard. The study quantifies verdict change rates across five reproducible SAS environments, finding them to range from 5.3% to 48.4% even with small shifts. To mitigate this, the researchers introduce the Freshness-Bounded Shield (FBS), a mechanism that estimates an approval's validity horizon. FBS achieves this by considering the approval's safe-side margin and recent feature volatility, without requiring an explicit plant-dynamics model. Using FBS, oracle-labeled approval-expiry rates were dramatically reduced from 3.4-24.7% to 0-1.8%. The study also found non-zero use-time invalidity in every approval stream audited by LLM judges. The authors conclude by formulating a "freshness contract," stipulating that every approval must be correct at both check time and use time.