Mechanistic Circuits Control Data Generation for AI Training

Current data synthesis methods often rely on heuristic prompt-based control, offering limited insight into how individual data samples influence a model's learning dynamics. To bridge this gap, a new circuit-grounded framework has been proposed, linking training-dynamics-based data valuation with mechanistic interpretability (MI). This framework aims to provide a white-box paradigm for interpretable data generation. The core of this approach involves uncovering specialized model-internal circuits that causally govern three complementary data utility signals: learnability, challenge, and alignment. Moving beyond traditional prompting, these identified circuits are then used as controllable interfaces to actively steer data generation, producing data specifically targeted for desired utility. Building on this capability, the researchers introduce SAMS (Stage-Aware Mechanistic Scheduling), which intelligently schedules circuit-steered data based on the model's evolving optimization needs. Experiments on multiple-choice QA tasks demonstrate that this approach generates precisely controlled data with greater diversity than prompt-based baselines, consistently leading to improved downstream performance and calibration. This work establishes MI not just as an analytical tool, but as a practical, controllable interface for data generation.