New Framework Boosts Trustworthy NL-to-Logic Translation.

Translating natural language instructions into formal, machine-interpretable specifications is vital for autonomous systems to plan, reason, and verify their behavior. However, current translation models often produce specifications for every input, regardless of reliability, posing significant risks in safety-critical applications. This paper addresses this challenge by proposing SCP-NL2TL, a selective translation framework.SCP-NL2TL not only generates formal specifications but also critically assesses their trustworthiness. It employs two complementary black-box signals to score reliability: the fidelity of the specification when back-translated into natural language, and the dispersion of multiple translations under semantic equivalence. These signals, which capture different types of errors, collectively provide a sharper distinction between correct and incorrect translations.The framework further integrates conformal risk control to calibrate this reliability score, enabling a decision to either accept a specification or abstain, with a statistically guaranteed bound on the rate of incorrect acceptances. An additional conformal anomaly detector pre-screens out-of-distribution inputs. This general framework, demonstrated across various temporal logic languages, establishes a foundation for more trustworthy natural language interfaces by empowering AI systems to recognize and manage their own uncertainties in critical translation tasks.