TeXFix-Bench Evaluates LLM Document Repair Capabilities

Scientific and technical writing heavily relies on markup languages like LaTeX, Typst, and Markdown, which must compile correctly. However, documents often fail to compile due to various errors such as missing delimiters, mismatched environments, or broken imports. Existing evaluations for LLM-based document repair typically use ad-hoc fault injection methods that lack a realistic error model, failing to capture the complexity of real-world compilation failures. To address this, TeXFix-Bench was developed as a multi-format benchmark for LLM-based full-source document repair. It is grounded in an empirically mined fault taxonomy derived from a Grounded-Theory study of real LaTeX hard-crash faults from sources like TeX Stack Exchange and GitHub. This taxonomy, instantiated as DocMut, includes 48 AST-aware operators across LaTeX, Typst, and Markdown. The benchmark contains over 10,000 instances from 743 open-licensed seeds. Evaluations of seven LLMs under a zero-shot protocol revealed that DocMut faults are significantly harder to repair than pattern-based mutations. A case study with 88 mined human crashes showed a 67% repair success rate. Crucially, a restoration oracle over 28,000 compiling repairs found that 13.6-18.5% materially altered the document text, indicating that compile success alone overstates repair quality. Typst documents were also found to be markedly harder to repair than LaTeX and Markdown.