Integrated Gradients Enhance Microbiome Transformer Explainability

Interpreting the decisions of complex AI models, particularly transformers applied to biological data like microbiomes, is crucial for trust and scientific discovery. In models like BiomeGPT, where input tokens fuse fixed species identities with variable abundances, traditional attention weights of the [CLS] token have been used for interpretation. However, these attention weights have significant limitations: they are non-negative, meaning they cannot differentiate between features that promote a disease state versus those that support health, and they operate after token fusion, obscuring the individual contributions of species identity and abundance. To overcome these shortcomings, this study advocates for the use of Integrated Gradients (IG), a signed and fusion-aware attribution method. The researchers propose a novel source-derived baseline for feature-tokenized models, which preserves species identity while isolating the impact of abundance variations. When applied to a disease-versus-health classification margin, IG yields polarity, explicitly identifying pathogenic or protective microbial signals. This gradient-based approach uncovers directional relationships between species and their abundances, along with sensitivity diagnostics that are entirely missed by unsigned attention weights. Furthermore, the paper recommends employing second-order Integrated Hessians to explore complex microbiome community interaction rules. This advanced technique can reveal how perturbing one microbial member alters the model's sensitivity to another, and which other species influence ambiguous cases towards disease or health at specific abundance levels. This work establishes a principled and more comprehensive approach to explainability for BiomeGPT and similar differentiable feature-tokenized transformers.