New Dataset Boosts Computer-Use Agent Training Performance

Training computer-use agents (CUAs) that interact with graphical interfaces requires extensive and varied data from real desktop environments. Existing large public datasets, such as AgentNet, have shown negative transfer effects when used for supervised fine-tuning, causing a decline in agent success rates. To address this, researchers developed ProCUA-SFT, a new dataset containing 3.1 million step-level samples. This data is derived from 93,000 synthetic trajectories generated across 2,484 different application configurations using a fully automated pipeline. The pipeline synthesizes grounded tasks on live desktops with real-world content and verifies task feasibility before execution. A single Vision-Language Model (VLM) acts as the goal generator, precondition judge, and trajectory executor, ensuring consistency. Fine-tuning models like UI-TARS 7B on ProCUA-SFT resulted in a substantial 18.7 percentage-point improvement on OSWorld tasks, significantly surpassing performance achieved with AgentNet. A portion of ProCUA also contributed to the capabilities of the Nemotron 3 Nano Omni model.