New Research Explores Geometric Limits of Neural Networks for Dynamic Programs.

Neural networks often face difficulties generalizing to longer inputs when tasked with solving dynamic programming (DP) problems. This research delves into the underlying geometric reasons for this limitation. It establishes that every finite min-plus DP can be understood as a shortest path problem on a Directed Acyclic Graph (DAG), which is mathematically equivalent to a tropical polynomial whose extended Newton polyhedron defines the decision boundaries. The paper formally proves that these three descriptions—the graph, the tropical polynomial, and the polyhedron—form isomorphic semirings at both the formal polynomial and computed function levels. These connections are characterized by operations that reveal structural redundancies within DPs. Crucially, the research then addresses the challenge of length generalization: whether the decision boundary learned for an input length *T* can predict the boundary for length *T+1*. It presents two significant structural negatives, demonstrating that the semiring's native dimension reduction methods are neither injective nor always closed within the DP context. Furthermore, standard series and parallel compositions cannot construct all DAG topologies from smaller sub-DAGs, and even comprehensive terminal-only operations fail to capture all DP compositions, highlighting fundamental barriers to length generalization.