Grok 4.5 Leads New Terminal Benchmark, AI Agents Still Struggle
Summary
A new benchmark for long-horizon terminal tasks, featuring 46 real-world challenges, has been released. Grok 4.5 currently tops the leaderboard, but even the best models only complete about half the tasks, indicating significant limitations in current AI agent capabilities.
Why it matters
This benchmark reveals the current limitations of AI agents in handling complex, multi-step tasks, providing a realistic assessment for professionals developing or deploying AI solutions. It indicates that while agents can initiate difficult work, they are far from being able to finish it reliably.
How to implement this in your domain
- 1Review the benchmark's methodology to understand the current state of AI agent performance.
- 2Evaluate existing AI agent solutions against these long-horizon task requirements.
- 3Prioritize research and development efforts on improving agent planning, memory, and error recovery for multi-step processes.
- 4Consider integrating human-in-the-loop systems for tasks where AI agents currently fall short.
Who benefits
Key takeaways
- A new benchmark assesses AI agents on 46 complex terminal tasks.
- Grok 4.5 leads the benchmark but only solves roughly half the tasks.
- Most tasks remain unsolved, indicating significant limitations in current AI.
- AI agents can start complex work but struggle with completion.
Original post by @minchoi
"New Long-Horizon-Terminal-Bench just dropped. 46 real terminal tasks. Hours of work. Hundreds of steps. And Grok 4.5 is #1 on leaderboard Still: nobody is close to solved. Best model only clears ~half mean reward. 29/46 tasks remain unsolved by anyone. Agents can start hard work.…"
View on X
Originally posted by @minchoi on X · view source
Want to go deeper?
Turn these trends into skills with Learnijoy's hands-on AI & tech courses.
Explore coursesMore in AI Engineering & DevTools
SQLite Recursive CTE Powers Novel Ray Tracer Implementation
A new project demonstrates a ray tracer built entirely using SQLite's recursive Common Table Expressions, storing its state on disk. This unique approach allows other tools to query the ray tracer's state in real-time, exemplified by a custom HTML+JS minimap.
Claude Tag Enhances Expressive Artifacts and Collaborative Dashboards
The "Claude Tag" feature now allows for more expressive artifacts that can be creatively combined, enabling users to build collaborative project dashboards. These dashboards can be edited by multiple users or through local "Claude Code" sessions.
Future Tools Website Receives Major Overhauls and Algorithm Improvements
The "Future Tools" website has undergone significant overhauls, introducing community-driven "Top 20" picks based on recent upvotes and an improved algorithm for recommending similar tools. The site also now features automatic deduplication and flagging of inactive tools, alongside enhanced curation to highlight valuable resources.