EcoAgent-Bench Evaluates LLM Agent Economic Decision-Making.

Traditional benchmarks for autonomous agents primarily focus on task completion, often treating resource consumption as a secondary metric. However, in real-world deployments, agents must make critical economic decisions, such as choosing between a cheap local lookup, an expensive broad search, or human escalation, all while adhering to a budget. This research addresses this gap by introducing EcoAgent-Bench.EcoAgent-Bench is a novel benchmark comprising 304 tasks derived from real-world scenarios, each specifying priced actions and an explicit budget. It challenges LLM agents to make four key economic decisions: avoiding unnecessary escalation, escalating when local evidence is insufficient, selecting an appropriate model tier, and knowing when to stop due to unsupported premises.Evaluations of seven LLM agents across tool-API and workspace-CLI settings revealed significant shortcomings. Agents achieved only 3.9-24.0% micro strict success, often either stopping prematurely or overspending on simple tasks. The study highlights that completion under a budget and economical action selection are distinct capabilities, with current LLM agents struggling to achieve economic consistency. This benchmark underscores the need for agents that can effectively balance performance with cost-awareness.