New Benchmark for Red Teaming LLM Agents and Measuring Safety

Large language model (LLM) agents, which combine reasoning with external tools, are susceptible to adversarial inputs that can lead to safety policy violations. Current evaluation methods often oversimplify agent safety into a single attack success rate, potentially obscuring the true nature of violations and the visibility of evidence. To address this, a new framework called REDAgentBench has been developed. REDAgentBench provides an executable environment for autonomous red-teaming and precise safety measurement. It generates attacks based on explicit safety constraints and known agent-system vulnerabilities, executing them within isolated service sandboxes. The framework then verifies harmful outcomes by analyzing service receipts and changes in the final state. Experiments with six models and three agent harnesses showed a macro-average attack success rate of 65.69%. The study also uncovered a "Recognition-Execution Gap," where nearly one in five confirmed violations occurred even after the agent acknowledged the relevant constraint or risk. A simple, training-free policy reminder was shown to reduce confirmed violations by over 70 percentage points in matched replays, highlighting actionable intervention points for improving agent safety.