VESTIGE Improves Genomic Transformer Accuracy on Degraded DNA.

Standard methods for fine-tuning masked-language models often assume that the difficulty of reconstructing masked tokens is uniform across all positions. This assumption breaks down when data degradation is concentrated at specific, predictable locations, leading to suboptimal model performance, especially in highly damaged regions. Researchers have introduced VESTIGE, a novel, parameter-free masking strategy designed to enhance the fine-tuning of genomic transformers. VESTIGE dynamically adjusts the masking distribution to align with the empirically observed per-position corruption profile of the input data, rather than applying a uniform masking rate. When applied to the challenging task of ancient DNA (aDNA) reconstruction, where cytosine deamination creates a predictable C-to-T/G-to-A gradient, VESTIGE demonstrated substantial improvements. It consistently outperformed standard masking across various damage levels, leading to higher reconstruction accuracy and significantly reducing false positive errors, even under extreme simulated damage. This principle is broadly applicable to any sequence data with measurable position- or context-specific corruption.