VeryTrace Verifies LLM Reasoning Traces for Accuracy.

This paper introduces VeryTrace, a novel framework designed to enhance the reliability of multi-step reasoning in large language models (LLMs) that use Chain-of-Thought (CoT) prompting. CoT reasoning can be fragile, as early errors or hallucinations can silently propagate, leading to incorrect conclusions. VeryTrace addresses this by formalizing natural-language reasoning traces into a structured, compilable representation using a Domain-Specific Language (DSL). This DSL explicitly defines step dependencies, mechanizes quantitative content, and structures semantic inferences. The framework employs a hybrid verifier that combines deterministic checks for computational correctness, dependency resolution, and constraint satisfaction with targeted LLM audits for non-mechanizable semantic judgments. This dual approach enables precise, step-level error localization and subsequent repair. Experiments across competitive mathematics, robotics planning, and kinship reasoning demonstrated that VeryTrace significantly improves accuracy over zero-shot baselines with state-of-the-art LLMs, without requiring domain-specific training or in-context examples, showcasing its precision and generalization capabilities.