DeepInsight Unifies Evaluation for Entire Physical AI Stacks

Evaluating a complete physical AI stack presents significant challenges due to the vast differences in operators, which can range from a single foundation model decoding step to thousands of physics ticks for whole-body control. Existing evaluation frameworks typically stitch together separate harnesses for each segment, leading to fragmented validity and making it difficult to diagnose regressions that span multiple layers. DeepInsight is introduced as a unified evaluation infrastructure capable of serving this entire spectrum on a single runtime. Instead of homogenizing these diverse regimes, DeepInsight maintains their heterogeneity by using three narrow, invariant abstractions: task, resource, and result. This means there's one episode driver, one resource-handle protocol for all expensive backends (like LLM inference and sandboxed runtimes), and one trace identity scheme for every event. This infrastructure has been deployed in production across all three layers of an embodied humanoid stack. It successfully reproduces published references and peer-framework readings for foundation-model evaluations, often running the same suites faster on a single node and scaling near-linearly across multiple nodes. DeepInsight's unique advantage lies in its diagnostic capabilities: because all layers write into a single shared trace, any regression originating in one layer but manifesting in another can be precisely localized on that trace, a capability that federated, per-segment harnesses cannot replicate.