Shopping Agents Learn from Online User Feedback

Large language model-based shopping agents are increasingly deployed in real-world e-commerce platforms, generating vast amounts of user interaction data. This data contains valuable implicit and explicit feedback that can be used to improve agent performance. However, existing approaches primarily rely on offline training signals, often overlooking the rich, yet heterogeneous, sparse, and noisy conversational feedback from users. To address these challenges, researchers propose LOFA (Learning from Online Feedback for Agents), a framework that allows shopping agents to learn directly from real online interaction logs without requiring human annotation. LOFA integrates reinforcement learning, which leverages verifiable purchase outcomes, with feedback-aware on-policy distillation. This distillation process identifies in-dialogue user directives and converts them into dense, token-level supervision. By combining these complementary objectives, LOFA effectively captures both collaborative behavioral patterns and individual user preferences, leading to consistent improvements in recommendation quality, response helpfulness, and alignment with user satisfaction in real-world e-commerce experiments.