Function-Aware FIM Boosts Coding Agent Performance.

Coding agents need to seamlessly integrate external tool outputs into their ongoing reasoning processes, a capability not fully developed by standard left-to-right code pretraining. This research identifies a structural similarity between a coding agent's action-observation-continuation loop and a function call site in code, where arguments are bound, a value is returned, and downstream code consumes it. Leveraging this insight, the authors propose function-aware fill-in-the-middle (FIM) as a mid-training objective. This self-supervised method masks functions selected via program dependency graph analysis and a complexity-inferability criterion. Applying this mid-training to models like Qwen2.5-Coder-Instruct and Qwen3-8B on a large decontaminated corpus resulted in significant performance gains on SWE-Bench-Verified (up to +3.2) and SWE-Bench-Lite (up to +5.4). Crucially, this approach also helps mitigate the common problem of capability erosion that agentic post-training often inflicts on non-agent coding tasks and other tool-use benchmarks, demonstrating a robust inductive bias even when trained only on Python code.