Large-Scale Autoregressive Pretraining Enables Controllable Catalyst Inverse Design

Designing heterogeneous catalysts remains a significant challenge due to their complex structures and the vast chemical space involved, which conventional screening struggles to explore efficiently. While machine learning has accelerated catalyst discovery, its efficiency diminishes as the search space expands, highlighting the need for generative models that can directly construct catalysts with desired properties. This research presents a novel conditional catalyst generative model, leveraging a Generative Pretrained Transformer (GPT) architecture augmented with a numerical embedding layer. This design enables the model to generate catalyst structures based on both categorical and continuous properties within a unified autoregressive framework. The model was extensively pretrained on 133 million catalyst structures and subsequently fine-tuned on approximately 460,000 optimized structures with associated properties. This resulted in a high structural validity of 98%, 95% optimization validity, and strong categorical condition fidelity. For binding energy conditioning, the model achieved a four-fold improvement over baseline distributions, leading to a 1.5 to 4-fold increase in screening efficiency for targeted catalyst discovery without further fine-tuning. This demonstrates a practical pathway towards controllable catalyst generation and accelerated discovery.