Affordance20Q Benchmarks LLM Affordance Reasoning from Physical Properties.

Affordance reasoning, which involves inferring an object's potential actions from its physical characteristics, is crucial for human understanding and increasingly important for Large Language Models (LLMs). Existing benchmarks often inadvertently allow models to rely on memorized object-affordance mappings by explicitly revealing object identities, rather than truly testing reasoning from physical properties. To address this, Affordance20Q is introduced as a novel benchmark. It frames affordance reasoning as a 20-Questions game where models must identify a hidden object's affordance by asking yes/no questions about its physical properties, without knowing the object's identity. The benchmark comprises over 1,000 games across hundreds of objects and affordances, all meticulously annotated. Experiments with 15 state-of-the-art LLMs show a substantial performance gap compared to humans, indicating models struggle with discriminating questions as the game progresses. To improve this, the researchers developed KB-Anchored Rule Induction (KARI), an LLM-based pipeline that generates affordance rules grounded in knowledge bases. KARI significantly boosts open-source LLMs' performance, though its gains are limited by knowledge base coverage.