New Method Improves Rare Event Estimation in Language Models

Quantifying the risk of rare failures in large language models (LLMs), especially under adversarial conditions or massive deployments, is crucial but challenging due to the extremely low probabilities involved. Traditional random sampling methods are inadequate for such rare events, and existing low-probability estimation pipelines often suffer from fragility, including zero-estimate collapse and systematic bias. This paper introduces a novel approach called Gradient Activation Adaptive Multi-Level Splitting (GA-AMLS). This method adapts rare-event Monte Carlo techniques to the continuous activation space of LLMs, using a gradient-based MCMC kernel. This innovation eliminates the zero-estimate problem common in input-space searches and replaces problematic independence assumptions with conditional sampling under a heavier-tailed activation prior. Additionally, the researchers propose the Shifted-Power Bregman (SPB) Loss, a proper scoring rule designed to remain stable even with zero-estimates and offering tunable asymmetry for underestimation versus overestimation penalties. Experiments on small transformer models show GA-AMLS achieving the lowest loss under symmetric evaluation, while highlighting that the optimal estimator choice depends on the specific deployment context and its associated cost asymmetries. This work establishes activation space as a viable domain for robust rare-event estimation in LLMs.