HDE-Net Improves Tabular DNNs with Hyperbolic Decision Modeling

Tabular classification problems often involve local, condition-triggered rules rather than smooth, global patterns. However, deep neural networks (DNNs) typically rely on Euclidean representations, which are better suited for smooth variations and semantic locality. This mismatch in geometric representation can make it difficult for tabular DNNs to effectively capture the discrete, rule-partitioned structures inherent in tabular data. To overcome this limitation, a new model called HDE-Net (Hierarchical Decision Embedding Network) is introduced. HDE-Net is a manifold-constrained DNN designed to enable hierarchical decision modeling within hyperbolic space. It abstracts heterogeneous features into unified Latent Decision Nodes (LDNs) and embeds them into the Poincaré ball, creating a continuous representation that mimics tree-structured reasoning. For numerical features, HDE-Net incorporates a Soft Decision Routing mechanism that approximates range-based local rules in a differentiable manner, aligning their LDN semantics with those of categorical features. An entropy-aware capacity allocation algorithm further optimizes the number of LDNs per numerical feature, balancing expressiveness and complexity. On the TALENT-tiny-core classification benchmark, HDE-Net achieved the best average rank, outperforming both industrial Gradient Boosted Decision Trees (GBDTs) and other recent tabular DNNs while maintaining high efficiency.