FM-Bench Evaluates AI Agents in Long-Horizon Management Tasks

While language model agents are becoming proficient at bounded tasks, their ability to make effective, sustained decisions over long periods, where actions have cumulative effects and the environment adapts, remains largely unmeasured. To address this, FM-Bench (Football Management Benchmark) has been introduced. FM-Bench simulates a football club manager's role over 20 in-game years, requiring an LLM agent to utilize 26 tools and make hundreds of decisions, including drafting, trading, contract negotiation, investment, and lineup setting. A deterministic engine scores performance without human or LLM judgment. The benchmark includes both solo play against a frozen world and an "Arena" where models compete head-to-head. All 15 frontier models tested successfully completed the 20-year horizon, unlike scripted baselines that often failed. Claude-fable-5 emerged as the top performer in both solo and Arena modes, though titles rotated among models in the Arena. Interestingly, model scale, price, or vendor did not predict performance. Instead, managerial behaviors like strategic investment timing and proactive contract renewals were key differentiators. Models struggled with learning hidden market prices and exhibited issues with self-managed memory, either growing archives indefinitely or rewriting plans every season.