SafeEvolve Improves AI Agent Safety Through Co-Evolution

The performance and safety of Large Language Model (LLM)-based agents are determined by both the underlying model and the "harness" that governs its interaction with the environment. This dual dependency exposes agents to safety risks, not just in their final responses but also throughout multi-step execution trajectories. Current safety alignment methods often focus on either external harness updates or internal policy optimization in isolation, failing to fully integrate runtime control with intrinsic safety. SafeEvolve proposes an innovative solution: an experience-driven, self-evolving framework for agent safety alignment. It operates on a continuous loop where safety experiences gathered from completed on-policy trajectories drive the co-evolution of both the agent's harness and its policy. On the harness side, SafeEvolve translates trajectory-level safety evidence into bounded, component-level updates for safety prompts and hierarchical skills, producing auditable and reversible harness artifacts. Concurrently, on the policy side, SafeEvolve employs a two-stage Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) paradigm. SFT first bootstraps the policy to actively utilize the evolved harness artifacts, while harness-augmented RL further refines autonomous safety behaviors during multi-step exploration using verifier-decomposed rewards. This co-evolutionary process effectively converts safety experience into an improved runtime harness and enhanced policy behavior. Experiments on agentic safety benchmarks demonstrate that SafeEvolve achieves a superior safety-utility tradeoff, for instance, reducing Attack Success Rate (ASR) by 3x on AgentDojo for Qwen3.5-4B while also improving benign utility.