AdaptRubric Enhances GUI Agents with Task-Adaptive Rewards

Tao Xiong, Xavier Hu, Wenkai Wang, Qinzhuo Wu, Changqiao Wu, Pengzhi Gao, Wei Liu, Jian Luan, Shengyu Zhang· August 26, 2026 View original

Key takeaways

  • AdaptRubric creates task-adaptive judging criteria for GUI agents.
  • It uses a coarse-to-fine approach for rubric construction.
  • The framework significantly improves GUI agent performance and task success.
  • This leads to more accurate and efficient automated GUI interactions.

Who benefits

Software TestingAutomationAI/ML EngineeringCustomer ServiceAccessibility

Summary

AdaptRubric is a new framework that improves GUI agent performance by generating task-adaptive judging criteria for reward modeling. It uses a coarse-to-fine approach to construct rubrics, retrieving category-level criteria and then generating instance-level specifics, leading to significant F1 and task success gains.

Recent advancements in GUI agents often rely on outcome reward modeling, where success is judged by how well a trajectory meets user instructions. However, existing GUI reward verifiers frequently lack task-adaptiveness, either using generic rubrics or implicitly reasoning, which can lead to overlooking specific constraints or enforcing unstated requirements. The AdaptRubric framework addresses this by introducing a Coarse-to-Fine Rubrics approach for constructing highly task-adaptive judging criteria. It begins with a category-level coarse stage, routing the instruction to a GUI task family and retrieving relevant reusable criteria. This is followed by an instance-level fine stage, which generates compact cues for concrete values, scopes, and constraints specific to the current instruction. This method ensures that the judging criteria are precisely tailored to each task instance. AdaptRubric consistently outperforms previous reward agents in both offline reward evaluation and online reinforcement learning optimization, demonstrating significant improvements in F1 scores and overall task success rates.

Why it matters

Professionals developing automated GUI agents or testing frameworks can leverage AdaptRubric to create more intelligent, accurate, and efficient systems that better understand and execute user instructions.

How to implement this in your domain

  1. 1Analyze current GUI agent reward modeling for adaptability and specificity.
  2. 2Implement a two-stage rubric construction process: category-level retrieval and instance-level generation.
  3. 3Develop a system to extract concrete values, scopes, and constraints from user instructions.
  4. 4Integrate AdaptRubric into existing reinforcement learning pipelines for GUI agents.
  5. 5Benchmark the performance gains in F1 score and task success against baseline methods.

Original post by Tao Xiong, Xavier Hu, Wenkai Wang, Qinzhuo Wu, Changqiao Wu, Pengzhi Gao, Wei Liu, Jian Luan, Shengyu Zhang

"arXiv:2608.24174v1 Announce Type: new Abstract: Recent studies on GUI agents have increasingly focused on outcome reward modeling, which assigns outcome rewards by judging whether an executed trajectory satisfies the success criteria implied by the user instruction. Existing GUI…"

View on X

Originally posted by Tao Xiong, Xavier Hu, Wenkai Wang, Qinzhuo Wu, Changqiao Wu, Pengzhi Gao, Wei Liu, Jian Luan, Shengyu Zhang on X · view source

Want to go deeper?

Turn these trends into skills with Learnijoy's hands-on AI & tech courses.

Explore courses