PolyWorkBench Benchmarks Multilingual Long-Horizon LLM Agents.

This research introduces PolyWorkBench, a novel benchmark specifically designed to evaluate the performance of Large Language Model (LLM) agents in multilingual, long-horizon workplace workflows. While LLM agents have demonstrated strong capabilities in tasks requiring planning, tool use, and environmental interaction, most existing benchmarks assume a monolingual context. Real-world applications, however, frequently involve processing and generating information in multiple languages within a single workflow, an area that remains largely unexplored in agent evaluation. PolyWorkBench comprises 67 diverse tasks spanning five critical domains: commerce, knowledge work, legal analysis, localization, and manufacturing. These tasks require agents to handle heterogeneous multilingual inputs, engage in iterative reasoning, invoke external tools, and produce structured outputs. To ensure a comprehensive assessment, the benchmark employs a hybrid evaluation framework that combines structural grading, executable verification, and LLM-based semantic assessment, capturing both functional correctness and linguistic consistency. Empirical results from testing state-of-the-art LLM agents reveal a significant performance drop in multilingual workflow settings compared to their monolingual counterparts. The analysis suggests that multilinguality introduces compounding negative effects across various reasoning and execution steps. This highlights a critical need for future research and development to jointly model language variation and procedural decision-making to improve LLM agent performance in globalized professional environments.