Scaling Laws Predict Particle Physics Model Performance.

Training large machine learning models in particle physics is computationally expensive, making it crucial to estimate performance before committing vast resources. This paper introduces a method for predicting the performance of transformer-based foundation models for collider jets using scaling laws. Unlike previous attempts, these scaling laws can forecast the loss of models not included in the initial fitting, even when trained with over a hundred times more compute. By fitting a joint model-and-data scaling law on smaller models, the researchers could predict the loss of much larger models to within one percent. Crucially, this predicted pretraining loss directly correlates with downstream physics performance, including fine-tuning loss and background rejection in standard tagging benchmarks. This breakthrough means that a compute budget can now be translated into expected physics performance before any large model is trained, enabling more efficient resource allocation and model design in high-energy physics. The study also released five pretrained models and the complete training recipe.