HERO Optimizes LLM-Generated Programs by Recombining Edits

This research introduces HERO (Heterogeneous Edit Recombination), a novel program optimizer designed to enhance LLM-driven program optimization. Existing methods often suffer from a "weakest-link effect," where a single poor edit within a bundle can negate the benefits of other good edits, hindering the search for optimal programs. HERO addresses this by moving beyond explicit textual "gradients" and instead prompts LLMs to generate diverse, non-overlapping atomic edits directly from the current program. The core innovation lies in systematically selecting and composing these individual edits into coherent program improvements, guided by evaluator scores. This zeroth-order optimization strategy allows for more effective exploration of the program space. Evaluations across various domains, including algorithmic problems, strategy games, agentic system design, and robotic path planning, demonstrate that HERO consistently discovers higher-scoring programs and converges substantially faster than previous LLM-based optimizers, while also being more token-efficient.