Memory-Efficient Transformer Improves Scalable Peptide Design

A new E(3) equivariant backbone, the Memory Efficient Equivariant Transformer (MEET), has been developed to address the scalability challenges in target-specific peptide design. This process requires simultaneously designing both sequence and structure under complex atomic geometric constraints. Existing latent generative frameworks are effective but often limited by the geometric backbone's scalability during encoding, decoding, and denoising. MEET tackles this by maintaining coupled invariant scalar and equivariant vector feature streams, while re-engineering geometric computations around memory-efficient attention. It incorporates global coordinate aggregation, pairwise distances, and covalent bond information to enhance its capabilities. Integrated into a VAE and latent diffusion pipeline, MEET achieves linear memory scaling with atom count and significantly improves the quality of generated peptides compared to current methods. Its ability to support systematic model and data scaling on large datasets promises better binding affinity, physical validity, and sample diversity.