Reasoning Effort Impacts LLM API Costs, Not Always Accuracy

When purchasing API access to large language models, users are essentially entering into a dated contract that specifies not just the model, but also terms like reasoning effort, output rails, and pricing. This research specifically examines the "reasoning-effort" term by comparing the performance and cost of a model (Sonnet 5) when explicitly requested to use high effort versus when effort was omitted. The study, conducted using AIME 2026 items, revealed that explicitly requesting "high effort" resulted in a higher delivered cost per API call. However, the research did not detect a statistically significant difference in accuracy between the high-effort and omitted-effort contracts. While the interval permits a potential accuracy gain of up to 4.67 percentage points, the design could not definitively confirm such a gain. The findings suggest that while paying for higher reasoning effort increases costs, it doesn't guarantee a proportional increase in accuracy for all tasks or models. This highlights the importance of understanding the specific API contract and model-specific semantics, as well as conducting empirical testing to optimize cost-performance trade-offs.