New Benchmark Reveals AI Agents Fail Hidden Social Norms

As multimodal large language models (MLLMs) are increasingly used as embodied agents in virtual or physical environments, their ability to navigate not just explicit goals but also implicit social norms becomes critical. This research highlights a significant gap in current MLLM capabilities by introducing NormAct, a new benchmark specifically designed to test compliance with hidden social norms during embodied planning. NormAct presents ordinary tasks where success requires both achieving an instructed goal and adhering to unstated social conventions. Evaluations of leading MLLMs (like GPT-5.4, Claude Opus 4.7, Gemini 3 Pro) revealed a stark disparity: while models achieved explicit goals in over two-thirds of cases, they complied with hidden norms in less than a third. This suggests the issue isn't a lack of general social knowledge, but rather difficulty in activating and applying relevant norms within a specific context. To address this, the paper proposes NormPerceptor, a context-conditioned cue generator that infers scene-relevant norms before planning. Integrating NormPerceptor significantly improved task success, nearly doubling it. This work underscores the necessity for embodied agents to proactively detect, ground, and integrate hidden social norms into their action planning to achieve truly appropriate and effective behavior.