Fair Tabular Foundation Models Address Bias

This paper addresses the critical issue of fairness in Tabular Foundation Models (TFMs), which are increasingly used in high-stakes decision-making but whose fairness properties remain largely unexplored. The authors propose FairTFM, a scalable training strategy designed to embed fairness constraints directly into TFM training, enabling fair predictions in a single forward pass. FairTFM tackles two main challenges: limited access to sensitive attributes in training data and the incompatibility of existing fairness techniques with in-context learning. It employs synthetic fairness tasks and a fairness-aware architecture featuring a gradient reversal layer. This encourages the model to learn representations that are invariant to sensitive attributes, thereby mitigating bias. Experiments across 132 fairness tasks demonstrate that FairTFM consistently improves fairness while maintaining competitive predictive accuracy, making it a significant step towards more ethical AI in tabular data applications.