New Research on Certifying LLM Outputs with Conformal Risk Control

A new study delves into the capabilities and limitations of Conformal Risk Control (CRC) for providing formal reliability guarantees for structured outputs generated by large language models (LLMs). These structured outputs include tasks like Named Entity Recognition (NER), JSON extraction, and classification, where current heuristic abstention policies often fall short of user-specified risk targets. The research presents an impossibility result, demonstrating that if an LLM's inherent base risk is too high, any distribution-free method like CRC will inevitably have to abstain on a significant fraction of examples. This provides a crucial feasibility test to determine if CRC is viable before implementation. The study further compares various certification bounds, including Hoeffding, empirical Bernstein, and a betting-based e-CRC, showing that more advanced bounds offer substantial gains, especially when calibration data is scarce. Finally, the paper validates Adaptive Conformal Inference (ACI) as a method to maintain certification reliability even when data distributions shift across datasets, significantly reducing risk-target violations. The findings culminate in a three-step deployment recipe for practitioners, offering guidance on checking feasibility, selecting appropriate bounds, and mitigating data shift.