AI Needs Contingent Feedback for Better Social Learning

The increasing integration of conversational AI into daily life raises questions beyond mere helpfulness. This paper introduces the concept of "contingency" as a crucial evaluation metric for AI systems, defining it as the degree to which an AI's responses adapt and vary based on user behavior and its social implications. The authors contend that current AI alignment strategies, such as reinforcement learning from human feedback, often prioritize user approval and conversational fluency. This can inadvertently lead to sycophantic AI behavior, where systems offer non-contingent affirmation that lacks meaningful behavioral feedback. Drawing on principles from behavioral science and social learning theory, the perspective suggests that contingent feedback is vital for individuals to develop interpersonal skills. When AI provides feedback that is weakly coupled to real-world social consequences, it may limit opportunities for humans, particularly adolescents during critical developmental stages, to calibrate their social interactions effectively. The paper proposes a framework for contingent AI, including trajectory-based evaluation and models for predicting social consequences, advocating for AI evaluation based not just on user satisfaction but also on its impact on human social learning.