TRACE Benchmark Diagnoses Human-AI Coordination Failures Under Drift

Modern AI-assisted and cyber-physical systems involve complex interactions between human operators, AI decision modules, and automated controllers. Ensuring trustworthiness requires understanding how drift and failures propagate across these layers, yet no standard benchmark exists to capture these multi-layer interactions. This paper introduces TRACE, a benchmark specifically designed to address this gap, focusing on drift. TRACE injects controlled drift into traces from the ALFRED benchmark, which covers everyday household tasks, creating 1,918 drifted traces. Each trace records step-by-step data across five layers (state, observation, decision, rules, control) and is labeled with drift type, affected layer, onset time, responsible actor, and causal mechanism. A baseline study using classical, recurrent, and attention-based models shows that drift is identifiable and attributable across all model families, indicating TRACE's effectiveness in diagnosing coordination issues.