New Transformer Model Generates Realistic Single-Cell Gene Expression Data

This research introduces a novel autoregressive transformer architecture designed for generating synthetic single-cell gene expression data. The model, which incorporates a learned quantized Variational Autoencoder (VAE) tokenizer, is trained to produce gene expression vectors that closely mimic real biological data from specific cell types. The study rigorously evaluates the model's ability to generate biologically faithful data and investigates its scaling properties by varying parameters and training data. A key finding is the identification of the first jointly-fit two-exponent scaling law and compute-optimal frontier for a single-cell foundation model. This work also suggests future applications, such as fine-tuning the pretrained model for predicting cellular responses to perturbations.