Grok 4.5 Leads New Terminal Benchmark, AI Agents Still Struggle
Key takeaways
- A new benchmark assesses AI agents on 46 complex terminal tasks.
- Grok 4.5 leads the benchmark but only solves roughly half the tasks.
- Most tasks remain unsolved, indicating significant limitations in current AI.
- AI agents can start complex work but struggle with completion.
Who benefits
Summary
A new benchmark for long-horizon terminal tasks, featuring 46 real-world challenges, has been released. Grok 4.5 currently tops the leaderboard, but even the best models only complete about half the tasks, indicating significant limitations in current AI agent capabilities.
Why it matters
This benchmark reveals the current limitations of AI agents in handling complex, multi-step tasks, providing a realistic assessment for professionals developing or deploying AI solutions. It indicates that while agents can initiate difficult work, they are far from being able to finish it reliably.
How to implement this in your domain
- 1Review the benchmark's methodology to understand the current state of AI agent performance.
- 2Evaluate existing AI agent solutions against these long-horizon task requirements.
- 3Prioritize research and development efforts on improving agent planning, memory, and error recovery for multi-step processes.
- 4Consider integrating human-in-the-loop systems for tasks where AI agents currently fall short.
Original post by @minchoi
"New Long-Horizon-Terminal-Bench just dropped. 46 real terminal tasks. Hours of work. Hundreds of steps. And Grok 4.5 is #1 on leaderboard Still: nobody is close to solved. Best model only clears ~half mean reward. 29/46 tasks remain unsolved by anyone. Agents can start hard work.…"
View on XOriginally posted by @minchoi on X · view source
Want to go deeper?
Turn these trends into skills with Learnijoy's hands-on AI & tech courses.
Explore coursesMore in AI Engineering & DevTools
Zapier vs. Tray: Enterprise Automation Platform Comparison for 2026
This post compares Zapier and Tray.io, evaluating which platform is better suited for enterprise automation needs by balancing power and ease of use. It argues that the best tools scale for complex requirements while remaining intuitive for all users.
Google Gemini Notebook Integrates with Play Books for AI Interaction
Google's AI note-taking app, Gemini Notebook, now features "Expert Intelligence," allowing users to import purchased Google Play Books and interact with their content, generating summaries, plans, infographics, and even AI podcasts.
Amazon Bedrock Adds OpenAI GPT-5.6 Models with India Cross-Region Inference
Amazon Bedrock now supports OpenAI's GPT-5.6 models, Terra and Luna, in India, enabling local data processing and inference within the region.