New Framework Tests Coding Agent Security in Software Pipelines

Coding agents are increasingly being integrated into software development and operations, where their ability to use tools can directly modify project artifacts, execution environments, and underlying systems. This presents a significant security challenge, as an agent's actions, not just its verbal output, can have lasting and potentially malicious effects. For instance, an agent could insert a persistent hook into a system startup script, abusing delegated privileges. To address this, a new execution-grounded red-team testing framework has been developed. This framework focuses on probing the execution-layer security boundary by observing sandbox evidence, including tool invocations, runtime traces, and file-system differences. It embeds target unsafe operations into common software engineering workloads like unit testing, regression testing, and validation. An execution oracle guides the refinement process when initial probes are rejected or fail. Experiments across multiple agent frameworks and model backbones show that reformulating red-team workloads to disguise risky intent within plausible engineering tasks substantially increases verified unsafe execution, reaching up to 73.61% for code carriers. This demonstrates that coding agents in system operations remain vulnerable when malicious intent is hidden, underscoring the urgent need for stronger security testing and safeguards.