FraudBench Stress-Tests Banking AI Agents Against Adaptive Fraud.

Conversational AI agents in banking are increasingly performing critical tasks like changing contact details or moving money, making their security against fraud paramount. Traditional fraud benchmarks focus on static transactions or generic prompt injections, failing to assess how agents handle sophisticated, conversational fraud attempts that manipulate identity, authorization, and trust. This research introduces FraudBench, a novel executable benchmark specifically designed to stress-test policy-grounded banking agents. FraudBench operates within a simulated banking environment, where both the agent and a simulated caller interact using tools and shared account states, with the agent referencing a 698-document internal policy corpus. The benchmark includes 150 adversarial scenarios, with 107 public tasks covering ten fraud mechanisms and chained adaptive attacks. Initial evaluations of four agents on these tasks showed attack-security rates between 49% and 65%, highlighting common vulnerabilities such as money-mule and first-party fraud. The benchmark emphasizes history-dependent safety, where a seemingly valid later request can become unsafe due to earlier conversational probes.