New Pipeline Boosts Low-Resource Code Generation for SLMs

Large Language Models excel at code generation for popular languages like Python, but their performance drops sharply for Low-Resource Programming Languages (LRPLs) such as Julia. Small Language Models (SLMs) face a challenge in this domain due to data scarcity for supervised fine-tuning, high inference costs, and limited benefits from reinforcement learning alone. Researchers have introduced a three-phase pipeline, "Selective Left-Shift," to address these issues by separating syntax acquisition from algorithmic reasoning. The first phase involves "left-shifting" inference-time computation to an offline data synthesis engine. This engine generates verified training examples by iteratively using compiler and test feedback. In the second phase, an SLM is fine-tuned on this synthetic, verified data to establish strong syntactic priors. Finally, the third phase applies Reinforcement Learning with Verifiable Reward (RLVR), which is grounded by language-agnostic Input/Output tests. The strong syntactic prior from the SFT phase helps constrain exploration, preventing syntax errors. This pipeline, applied to Qwen3-8B, achieved significant improvements in pass@1 scores on MultiPL-E and Agnostics LiveCodeBench for Julia, outperforming state-of-the-art results with substantially less data and cost. The method also generalized effectively to Ballerina, a language with minimal pretraining representation.