IMBench Benchmark Evaluates Intuitive Robotic Manipulation Capabilities

A new benchmark called IMBench has been introduced to assess the intuitive manipulation capabilities of robots, a critical area where current AI models often fall short. Unlike existing benchmarks that isolate physical reasoning or policy performance, IMBench evaluates the integrated ability of robots to combine perception, physical reasoning, action generation, and iterative execution to solve complex tasks. This integrated approach mirrors how humans intuitively interact with the physical world. The benchmark comprises 35 diverse tasks, including scenarios with contact-rich manipulation, tool use, and multi-stage dependencies, along with 14,000 filtered trajectories and scalable tools for scenario generation. Initial experiments using IMBench have highlighted a consistent performance gap: while vision-language models show some physical reasoning, they struggle to produce executable plans. State-of-the-art vision-language-action models, on the other hand, face difficulties in satisfying task constraints and generalizing across different scenarios. This research positions IMBench as a crucial step towards developing and evaluating more integrated and adaptive physical intelligence in robotics.