Automated Red-Teaming Synthesizes Hard Examples for MLLM Robustness.

Multimodal Large Language Models (MLLMs) are increasingly used for sensitive tasks like content moderation, but they remain vulnerable to adversarial attacks and unusual edge cases. Traditional methods for finding these vulnerabilities, such as active learning or manual annotation, struggle to keep pace with the complexity and volume of new threats. This research proposes an innovative, automated red-teaming framework that uses an agentic approach to systematically generate challenging examples. The framework operates with a multi-agent architecture, including a high-reasoning "Architect" agent, an advanced image generator, and a multi-level committee of LLM raters for verification. This system autonomously identifies boundary-pushing violations and ambiguous policy scenarios. By using these synthesized adversarial examples as in-context demonstrations during testing, the target MLLM's robustness is substantially improved, evidenced by a significant reduction in False Negative Rate on a public image safety benchmark, all without requiring human labeling.