AnTrap Benchmarks Android GUI Agent Robustness to Anomalies.

Android GUI agents frequently encounter unexpected runtime anomalies, such as sudden pop-ups or misinterpretations of actions, when deployed on real devices. However, existing benchmarks lack a systematic way to evaluate agent robustness against these dynamic perturbations. This paper presents AnTrap, a comprehensive benchmark designed to inject realistic adversarial conditions into agent execution trajectories. AnTrap categorizes real-world anomalies into a four-layer taxonomy (State, Thinking, Action, and Round) with ten fine-grained subcategories, ensuring task solvability while introducing adversarial elements. Evaluating 16 prominent GUI models, the study uncovered a universal susceptibility to dynamic anomalies, with even the best models experiencing significant performance degradation. The research also used GRPO training to distinguish between environment-learnable anomalies and those stemming from fundamental reasoning bottlenecks, concluding that while some single-step traps can be addressed through adversarial reinforcement learning, deeper contextual traps like state deadlocks expose inherent limitations.