ELMER: Evolutionary LLM for Program Search and Refinement

In program evolution, assessing whether a code mutation is beneficial is straightforward, but controlling the behavioral impact of that mutation is difficult. Small syntactic changes can lead to drastic behavioral shifts, while large rewrites might preserve execution traces. This paper introduces ELMER (Evolutionary Language Model that Explores and Refines), a novel approach that leverages language models for program search and refinement. ELMER operates by searching through natural-language descriptions of policies and then compiling these descriptions into typed programs for execution. At its core is a fine-tuned Qwen3-8B model, which learns three key task-conditioned operations: conditional semantic mutation, natural language to domain-specific language (GPTL) compilation, and GPTL to natural language translation. The model is fine-tuned using Direct Preference Optimization (oDPO), with conditional input specifying the desired mutation strength (low, medium, high). Across 252 fixed-budget evolutionary searches, the oDPO fine-tuning significantly improved both behavioral calibration and the efficiency of the search process. Natural language descriptions achieved the highest observed held-out fitness. Analysis revealed that the conditional input systematically alters the composition of semantic edits, and language-based mutations are better at preserving parent fitness for matched small-to-moderate behavioral displacements. These results highlight the potential of language as a steerable, execution-grounded representation for exploring and refining executable program spaces.