Code LLMs Struggle with Self-Correction via Uncertainty Signals.

Researchers investigated whether uncertainty estimation methods, typically used in natural language processing, could effectively enable self-correction in large language models (LLMs) for code generation. They evaluated five different uncertainty methods across three small code LLMs on standard benchmarks like HumanEval and BigCodeBench. The study found that while multi-sample P(True) showed the strongest correlation with correctness, most uncertainty-based self-correction policies, including adaptive decoding and uncertainty-based regeneration, failed to reliably improve code accuracy and often degraded performance. Significant gains in Pass@1 were only observed with verification-based self-correction, which relies on executing the generated code. This suggests that simple uncertainty signals alone are insufficient for improving code correctness and are best used as gates for more costly, execution-based verification loops.