SearchEyes Boosts Multimodal Deep Search Intelligence with Simulation

Zhengbo Jiao, Yiming Cheng, Yilei Jiang, Kaituo Feng, Rui Huang, Tianyi Jiang, Juanxi Tian, Jiapeng li, Qunzhong Wang, Tailai Chen, Qianshan Wei, Chuan Xiao, Shanyu Rong, Yangfu Li, Yanhan Zhou, Yunpu Ma, Yifan Zhang, Xiangyu Yue· July 8, 2026 View original

Summary

SearchEyes introduces a simulated search world, built on a typed knowledge graph, to unify training data, environments, and reward signals for multimodal search agents. Its Perception-Knowledge Chains and Hop-Anchored Policy Optimization enable state-of-the-art performance in multi-hop reasoning across multimodal knowledge-intensive benchmarks.

Training multimodal search agents to perform complex, multi-hop reasoning has been difficult because existing development pipelines often treat training data, search environments, and reward signals as separate components. This leads to discarded structural metadata, reliance on external search engines, and sparse rewards for reinforcement learning. To overcome these issues, SearchEyes proposes a novel approach using a "simulated search world" grounded in a typed knowledge graph. This unified framework integrates all three components. It introduces Perception-Knowledge Chains (PKC) to sample constrained multi-hop paths over the visual-knowledge intersection of Wikidata5M, preserving crucial hop-level entity metadata. This metadata simultaneously defines a self-contained search world and provides precise step-level reward anchors. Furthermore, SearchEyes incorporates Hop-Anchored Policy Optimization (HaPO), which directly reuses these anchors for step-level credit assignment, eliminating the need for a separate reward model. Experiments across six multimodal knowledge-intensive benchmarks demonstrated that SearchEyes achieves state-of-the-art performance among open-source multimodal search agents, with its 27B model improving significantly over the strongest baseline.

Why it matters

For professionals developing advanced search, recommendation, or knowledge retrieval systems, SearchEyes offers a breakthrough in training more intelligent and efficient multimodal agents capable of complex reasoning. This can lead to more accurate information discovery and enhanced user experiences.

How to implement this in your domain

  1. 1Evaluate existing multimodal search or recommendation systems for limitations in multi-hop reasoning and reward sparsity.
  2. 2Explore the SearchEyes framework for building simulated search worlds based on internal knowledge graphs or structured data.
  3. 3Implement Perception-Knowledge Chains to generate rich, metadata-anchored training data for multimodal agents.
  4. 4Apply Hop-Anchored Policy Optimization to improve credit assignment and training efficiency for multi-hop tasks.
  5. 5Benchmark SearchEyes against current search intelligence solutions to identify areas for performance improvement.

Who benefits

E-commerceContent PlatformsResearch & DevelopmentData AnalyticsCustomer Service

Key takeaways

  • SearchEyes improves multimodal search agents for multi-hop reasoning.
  • It uses a simulated search world based on a typed knowledge graph.
  • Perception-Knowledge Chains provide rich, step-level reward anchors.
  • Hop-Anchored Policy Optimization enhances training efficiency and performance.

Original post by Zhengbo Jiao, Yiming Cheng, Yilei Jiang, Kaituo Feng, Rui Huang, Tianyi Jiang, Juanxi Tian, Jiapeng li, Qunzhong Wang, Tailai Chen, Qianshan Wei, Chuan Xiao, Shanyu Rong, Yangfu Li, Yanhan Zhou, Yunpu Ma, Yifan Zhang, Xiangyu Yue

"arXiv:2607.05943v1 Announce Type: new Abstract: Training multimodal search agents to perform multi-hop reasoning remains challenging due to a fundamental structural disconnect: existing pipelines construct training data, search environments, and reward signals independently, caus…"

View on X

Originally posted by Zhengbo Jiao, Yiming Cheng, Yilei Jiang, Kaituo Feng, Rui Huang, Tianyi Jiang, Juanxi Tian, Jiapeng li, Qunzhong Wang, Tailai Chen, Qianshan Wei, Chuan Xiao, Shanyu Rong, Yangfu Li, Yanhan Zhou, Yunpu Ma, Yifan Zhang, Xiangyu Yue on X · view source

Want to go deeper?

Turn these trends into skills with Learnijoy's hands-on AI & tech courses.

Explore courses