New Benchmark Reveals LLM Agents Lack Long-Term E-commerce Coherence

While Large Language Model (LLM) agents are increasingly evaluated for their ability to use tools and complete bounded tasks, real-world applications often demand "Long-Term Coherence"—the capacity to maintain purposeful behavior over extended periods and adapt decisions based on accumulated evidence. To address this gap, a new benchmark called MerchantBench has been developed, focusing on seller-side e-commerce operations. MerchantBench is a 365-day, order-level simulation built upon nearly 100,000 real e-commerce product records. It provides a persistent environment where agent actions have future consequences, feedback arrives with varying delays, and incoherent behavior leads to measurable cumulative effects. The simulation covers critical e-commerce decisions such as product sourcing, listing and pricing control, cash-flow management, and adapting to mixed-latency feedback. The study evaluated eight different LLMs using two agent frameworks across 48 simulated runs. The results revealed a significant performance disparity between even the most advanced LLMs and human participants. The top-performing LLM configuration managed to achieve only 27.3% of the mean final net assets generated by human players, highlighting that current LLM agents still have substantial limitations in tasks requiring sustained strategic coherence and adaptive decision-making over long horizons.