PACE Optimizes Training for Iterate-Averaged LMs.

Many modern language model (LM) training pipelines do not return the final model iterate but rather an averaged version, such as an exponential moving average (EMA) of the training iterates. This practice raises a fundamental question: how can the training process itself be optimized to specifically enhance the performance of this averaged model? This research addresses this by framing optimizer design for the iterate-average estimator as an optimal-control problem. By solving this problem in a continuous-time stochastic quadratic model, the authors derive a control strategy that minimizes the error of the returned average while penalizing intervention size. A practical approximation of this controller leads to PACE, a lightweight wrapper for AdamW. PACE functions by pulling the live model weights towards their exponential moving average using a clipped, per-coordinate control strength. Theoretical proofs show that a stylized PACE converges efficiently, and in quadratic settings, it can substantially improve the limiting squared error of the iterate-average estimator. Empirical results demonstrate that PACE outperforms standard AdamW and EMA-evaluated AdamW in supervised fine-tuning of 1-2B parameter LMs and in GPT-2 pretraining across various hyperparameters.