Deployment Rules Critically Impact Multi-Agent AI Safety, Not Just Models.

The paper introduces a new evaluation method called "institutional red-teaming" for multi-agent AI systems. This approach focuses on assessing the impact of deployment rules on collective AI behavior and safety, rather than solely on the underlying AI models. By keeping agents, objectives, and task states constant and only varying a single rule, researchers can directly attribute changes in collective behavior to that specific rule. The methodology was applied using IABench-CA, a benchmark featuring 228 contexts, five canonical rules, and seven model populations, generating over 33,000 games. Key findings reveal that deployment rules significantly affect collective safety; for instance, altering a consequence rule can shift mean fatality rates by 22 to 58 percentage points within any given population. The study also highlights that there is no universally safe default rule, and the safest or least-safe rule, as well as the direction of its effect, varies across different AI populations. However, a consistent finding is that regressive identity-targeting rules are never decisively safest, often leading to the elimination of the least-resourced agent in a high percentage of games. The mechanism behind this is identified as identity salience, where merely naming the loss bearer in a rule text can drastically increase targeted elimination, even at identical payoffs. This research provides a safety-case workflow for certifying rule regions and identifying residual risks.