Tydra: Hybrid AI Model Boosts Tabular Data Efficiency.

This research introduces Tydra, a novel hybrid AI architecture specifically designed for processing tabular data efficiently. Existing Transformer-based models, while powerful for tabular tasks, suffer from quadratic computational costs as the context length increases. Conversely, purely State Space Model (SSM) alternatives offer efficiency but often at the expense of accuracy. Tydra aims to strike a balance by interleaving attention layers from Transformers with SSM layers. This hybrid approach allows it to leverage the strengths of both architectures. Evaluations across 30 OpenML datasets show that Tydra achieves a 30% reduction in inference time compared to TabPFN, a leading Transformer-based model, while largely preserving its predictive accuracy. Furthermore, Tydra outperforms a significantly larger Hydra model, an SSM-based alternative, while also providing faster inference. These results highlight the potential of hybrid architectures for developing more efficient and performant tabular foundation models.