AI Evaluation Needs Psychological Competence for Human-Facing Systems

While existing AI evaluation frameworks primarily concentrate on technical metrics like accuracy and robustness, they fall short when assessing AI systems designed for direct human interaction. These human-facing AIs, often acting as advisors or companions, significantly influence user reasoning, emotional interpretation, beliefs, trust, and decision-making. Therefore, the true unit of evaluation should extend beyond the model itself to encompass the entire human-AI interaction. This paper proposes "psychological competence" as a vital, yet overlooked, dimension for evaluating such systems. Psychological competence is defined as an AI's ability to appropriately support user cognition, emotional understanding, and behavioral decision-making, considering the user, context, and interaction purpose. This includes aspects like framing, tone, perceived authority, and how uncertainty is handled. The authors outline a conceptual framework for psychological competence, suggesting it can be assessed through scenario-based probes, structured human evaluations, and model-assisted methods, rather than a single benchmark. They advocate for its integration into evaluation practices for model providers, deploying organizations, researchers, and regulators to better understand the real-world impact of human-facing AI.