LLM-Solver Loops Face "Narration Gap" Risk, Compromising Verified Conclusions

Hybrid reasoning pipelines that combine Large Language Models (LLMs) with formal tools like SAT and SMT solvers are increasingly used for safety and security-critical questions. While solvers provide sound and verifiable answers, this research highlights a critical vulnerability: the "narration gap." This gap occurs when the LLM translates the solver's formal output into a user-understandable answer, potentially compromising the soundness guarantee. The study models the entire LLM-solver loop as a verified decision procedure, analyzing its three components: formalizing the question, deciding it with the solver, and narrating the result. Previous work focused on formalization and decision, but this paper specifically addresses the narration phase. Empirical evaluations on five open-source models under prompt injection attacks reveal that while certificate gating can ensure the solver's verdict remains sound, an adversary can still manipulate the LLM to invert a verified conclusion through various phrasings and communication channels. Although hardened prompts can significantly reduce injection attacks, they cannot eliminate them and remain vulnerable to adaptive attacks. The findings demonstrate that even with formal verification at the solver level, the robustness of the final answer presented to the user can be compromised due to the LLM's narration.