AIChilles Uncovers Hidden Weaknesses in AI-Evolved Software Systems

The computer systems community is increasingly adopting AI-driven evolution, where AI agents iteratively modify and improve systems. While this approach has shown promising results, such as significant performance improvements, there are concerns about the reliability of these AI-evolved programs, particularly regarding their performance on unseen workloads and potential scalability regressions. To address these concerns, a new system called AIChilles has been developed. AIChilles is designed to automatically uncover hidden weaknesses in AI-evolved systems. It takes a baseline program and its AI-evolved counterpart as input, then systematically searches for specific workloads that cause the AI-evolved program to perform worse than the baseline in terms of correctness, execution time, memory consumption, or output quality. AIChilles employs a combination of techniques, including deterministic workload-parameter extraction, agent-based constraint inference, differential oracles, and code-frequency coverage, to identify diverse types of failures. Across various system applications and numerous AI-evolved programs, AIChilles successfully discovered 49 distinct hidden weaknesses. The research also suggests that integrating AIChilles into the AI-driven development lifecycle can help mitigate these identified vulnerabilities.