CATTLE Enables Transfer Learning for Disjoint Tabular Data

Applying transfer learning to tabular data is notoriously difficult due to the inherent heterogeneity of features, structures, and semantics across different datasets. Existing methods typically assume shared features between source and target domains, a condition rarely met in real-world scenarios. This new research introduces a solution called CATTLE (Cross-domain Attention Transfer Learning) to overcome this limitation. CATTLE's innovation lies in its ability to learn "generalized context" using transformer projection weights for key, value, and query. Unlike conventional transformer activations that learn domain-specific context, these projection weights capture rule-based generalization. This allows the key projection weights from a source domain to interact effectively with the query weights in a target domain, facilitating knowledge transfer even when features are entirely disjoint. Extensive experiments across ten pairs of disjoint source-target datasets demonstrate CATTLE's superior performance. It consistently outperforms nine state-of-the-art baselines, including various machine learning, deep learning, and large-scale pre-trained transfer learning models. CATTLE achieved the best average rank and delivered a 3.7% average AUROC gain, proving its effectiveness in data-agnostic cross-domain transfer learning for tabular data.