AI-Designed Methods: Performance Matches Human, Designs Imitate

Recent advancements in large language model (LLM) agents have enabled them to design methods for complex AI tasks. This paper explores two key aspects of these agent-designed methods: their performance compared to human-designed counterparts and the originality of their algorithmic designs. The research introduces an analysis framework that maps both human and agent-designed methods into task-specific algorithmic design spaces, allowing for quantification of their differences at a module level. Various LLM agents were tested across a range of open-ended AI tasks spanning multiple modalities. Results indicate that while agents can sometimes achieve or exceed human state-of-the-art performance in specific configurations (10 out of 72), this success isn't consistently generalizable. Crucially, nearly all (96.8%) agent-designed methods fall within existing human-derived design spaces, primarily reusing and recombining known algorithmic choices, with almost half being exact matches to existing human designs. This suggests current AI excels at recombination rather than true algorithmic innovation.