Tabular In-Context Learners Predict Biomolecular Properties

Researchers investigated the surprising effectiveness of tabular in-context learning models, such as TabPFN3 and TabICL, for predicting biomolecular properties. These models are typically trained on synthetic tabular data derived from random causal graphs, a domain seemingly unrelated to the complex structures of proteins and molecules. Despite this, the study found that these models can indeed generalize well to biomolecular tasks. The research evaluated these models across two key areas: protein fitness regression and small-molecule classification. For protein fitness, using a fixed ESMC representation, tabular in-context learning consistently performed competitively. In the case of small-molecule classification with ECFP/RDKit descriptors, no single model pairing dominated, emphasizing that the choice of molecular representation is a critical factor, often more so than the tabular predictor's inherent bias. The conclusion is that tabular foundation models are strong contenders for biomolecular prediction, provided they are coupled with suitable sequence or molecular representations.