Agents Should Help Users Construct Preferences, Not Just Elicit Them.

This research challenges the conventional assumption that AI agents interact with "expert users" who possess fully formed and clear preferences. The paper argues that in reality, users often lack the necessary domain knowledge to articulate precise preferences, especially when a task is underspecified. Instead of merely asking clarifying questions, agents should actively assist users in *constructing* their preferences by offering relevant domain knowledge, examples, or explanations. To formalize this concept, the authors introduce CoPref, a model grounded in the Search-Experience-Credence framework from Information Economics, which describes how users build preferences through agent dialogue actions. They then present CoShop, an interactive benchmark specifically designed for agentic recommender systems. In CoShop, an agent engages in conversation with a CoPref user, and its success hinges on its ability to help the user gain the knowledge needed to specify their task effectively. Evaluating five frontier language models on CoShop, the study found that none of the agents achieved more than 56% accuracy, even after five turns of interaction. The primary reason for these failures was not the agents' inability to find suitable items, but rather their limited success in expanding the users' understanding of what they truly desired. This highlights a significant gap in current agent design, suggesting a need for agents that are more proactive in user education and preference formation.