AI Probes Predict LLM Agent Failure Early

This research introduces a novel method to predict and abort failing Large Language Model (LLM) agent trajectories early, significantly reducing wasted computational resources. The technique involves lightweight, per-round probes that analyze the agent's hidden internal representations, identifying signs of impending failure much earlier than methods relying solely on observable behavior. The core innovation is a recall-controlled probe cascade, a series of calibrated gates that ensure a user-specified global success rate for episodes. Applied to two different LLM agents on the TextCraft task, this cascade saved between 37.2% and 47.1% of inference compute at a 90% recall target, demonstrating its efficiency and effectiveness. The study also highlights that internal states are more predictive of failure than external behavior.