Runtime Observability for Diverse Attention Memory in LLMs.

Modern large language models (LLMs) no longer rely solely on simple KV caches; they employ diverse forms of attention memory, including latent caches, learned sparse selectors, and recurrent states. Each of these memory types behaves differently under compression and can fail in unique ways. This research proposes a novel runtime observability contract designed to cover all four memory classes using just three operators. The contract allows for the composition of per-stage bounds into an executable, request-level risk ledger. This system quantifies the trade-offs in memory usage and holds risk budgets with zero violations, even under high concurrency. A key feature is its ability to precisely localize silent corruptions within a served model stack, as demonstrated with a DeepSeek-V4 prototype. The entire system, including artifacts and formal proofs, is open-sourced, allowing for verifiable and auditable memory management.