Wrong Design Intent Headers Harm CAD Program Completion in LLMs

A study investigating the use of lightweight design-intent headers to guide parametric CAD generation by fine-tuned code Large Language Models (LLMs) has uncovered a critical finding: providing incorrect design intent can be more detrimental than providing no intent at all. While headers are intended to steer the LLM's output, this research rigorously tested whether the model genuinely interprets the header's content, rather than merely reacting to its presence. Using CADCON, a system that prepends a five-feature design-intent header to CadQuery-style programs, the researchers fine-tuned Qwen2.5-Coder-1.5B. The generated B-rep solids were then re-scored using executable geometric assertions, ensuring an independent metric from the header definition. The experiments, which included correct, wrong, and masked headers across different prefix lengths, showed that a semantically incorrect header significantly reduced the model's adherence to design intents it could otherwise render unconditioned. Crucially, a "derangement control" experiment, where the model was retrained with shuffled ground-truth headers (preserving marginal distribution but destroying content correlation), demonstrated immunity to wrong headers. This confirmed that the observed harm is not a simple distribution shift but arises from the learned mapping between the header and the program generation. The study also highlighted that independent metrics deflate the apparent benefit of correct headers, quantifying metric circularity, and that the harm is regime-specific, particularly evident when the model has some initial capacity to generate valid CAD.