# AIChilles: Automatically Uncovering Hidden Weaknesses in AI-Evolved Systems

**Authors:** Yajie Zhou, Ao Li, Ashwin Silla, Zaoxing Liu, Vyas Sekar  
**arXiv:** [2606.15834](https://arxiv.org/abs/2606.15834)  
**Date:** 2026-06-14  
**Summary page:** https://www.summarizepaper.com/en/paper/2606.15834v1/

## Key points

- The computer systems community is increasingly interested in AI-driven system evolution
- AI agents are used to iteratively rewrite systems for performance improvement
- Frameworks like AdaEvolve and Engram have shown significant score improvements (12-60%) over human-designed algorithms
- Concerns exist about the performance of AI-evolved programs on unseen workloads and scalability regressions
- AIChilles is a new tool developed to uncover hidden weaknesses in AI-evolved systems by comparing baseline program $P$ with AI-evolved program $P'
- AIChilles uses deterministic workload-parameter extraction, agent-based constraint inference, differential oracles, and code-frequency coverage to discover diverse failures across different system applications and AI-evolved programs
- In testing across five system applications and 30 AI-evolved programs, AIChilles identified 49 distinct hidden weaknesses
- Design requirements for detecting these weaknesses include general design compatibility with various target programs, high coverage across different weakness types, discriminative detection of weaknesses under adversarial workloads, and diversity in uncovering distinct weaknesses
- Traditional bug-finding techniques like fuzzing and symbolic execution may not be as comprehensive as needed for identifying hidden weaknesses in AI-evolved systems
- Including tools like AIChilles in the development lifecycle of AI-driven systems can proactively mitigate several of these weaknesses

## Summary

The computer systems community has shown a growing interest in AI-driven system evolution. This involves using AI agents to iteratively rewrite systems in order to improve performance. Frameworks such as AdaEvolve and Engram have reported significant score improvements ranging from 12-60% over human-designed algorithms. However, there are concerns regarding the performance of these AI-evolved programs on unseen workloads and potential scalability regressions. To address these concerns, a new tool called AIChilles has been developed. It automatically uncovers hidden weaknesses in AI-evolved systems by taking a baseline program $P$ and an AI-evolved program $P'$ as input. It then searches for valid workloads where $P'$ regresses relative to $P$ in terms of correctness, runtime, memory usage, or output quality. AIChilles combines deterministic workload-parameter extraction, agent-based constraint inference, differential oracles, and code-frequency coverage to discover diverse failures across different system applications and AI-evolved programs. In testing across five system applications and 30 AI-evolved programs, it identified 49 distinct hidden weaknesses. The design requirements for detecting these weaknesses include the need for a general design that can work with various target programs consuming inputs differently, high coverage across different weakness types beyond crashes or performance regressions, discriminative detection of weaknesses that reveal significant gaps between programs under adversarial workloads, and diversity in uncovering distinct weaknesses instead of repeatedly triggering the same failure mode. Traditional bug-finding techniques like fuzzing and symbolic execution are effective at uncovering specific classes of weaknesses but may not be as comprehensive as required for identifying hidden weaknesses in AI-evolved systems. By explicitly including tools like AIChilles in the development lifecycle of AI-driven systems, it is possible to mitigate several of these weaknesses proactively.