LOGICA Enhances Biological Language Models with Contextual Alignment

Pretrained biological language models typically provide per-token probability distributions, which are essential for tasks like sequence design and variant scoring. However, these distributions are often learned from general, unlabeled data and lack conditioning on specific biological contexts, such as cellular environments or drug interactions. Existing methods to introduce context often compromise the model's original interface by using pooled embeddings or separate prediction heads. Researchers have introduced LOGICA (Logit-space Contrastive Alignment), a novel framework that performs contrastive learning directly within the output-logit space. LOGICA utilizes gated cross-modal adapters to maintain the model's native likelihood interface, converting contextualized token log-likelihoods into matching scores. This approach allows learning from limited paired data across models with different vocabularies, proving particularly effective for mutation-local variant ranking and outperforming prior state-of-the-art methods in various biological prediction tasks.