COSI-Lab Dataset Models Multi-Perspective Social Intentions in Conferences

Zonghuan Li, Litian Li, Arthur Mercier, Gara Dorta, Balint Dioszegi, Jose Morales-Vargas, Chenxu Hao, Ivan Kondyurin, Vanessa Begemann, Nale Lehmann-Willenbrock, Bernd Dudzik, Saunaq Chakrabarty, Sotiris Vacanas, Laura Cabrera-Quir\'os, Anne L. J. ter Wal, Vitaliy Popov, Jorge Castro-God\'inez, Chirag Raman, Stephanie Tan, Hayley Hung· August 3, 2026 View original

Key takeaways

  • COSI-Lab is a unique multimodal dataset for modeling multi-perspective social intentions in real-world conference settings.
  • It treats subjective perceptions of intention as explainable reasoning, not just noise.
  • The dataset supports "Apparent Intent Inference" (AII) and provides benchmark tasks.
  • It combines multi-sensor data with self-reported goals for comprehensive social interaction analysis.

Who benefits

RoboticsHuman-Computer InteractionSocial AnalyticsAI DevelopmentMarket Research

Summary

COSI-Lab is a new multimodal, multi-sensor dataset capturing ecologically valid social interactions of 32 academics at a scientific workshop. It aims to model multi-perspective social intentions, treating subjective perceptions as explainable reasoning processes rather than noise.

Understanding human social intention is crucial for developing truly intelligent AI systems, but existing approaches often struggle with the subjective and multi-faceted nature of human perception. The COSI-Lab project introduces a unique dataset designed to address this challenge. It captures rich, multimodal, and multi-sensor data from 32 academics interacting in a weakly scripted, real-world scientific workshop environment, including mingling sessions with genuine professional and social stakes. The core innovation of COSI-Lab is its focus on modeling "Apparent Intent Inference" (AII) from the perspective of ex-situ observers, recognizing that intentions are independent of manifest outcomes. The dataset features a novel annotation process that accounts for perceivers' interpretive tendencies, providing quantitative and qualitative analyses of intent narratives. It also offers benchmark tasks for AII, speech quality audio, privacy-preserving multimodal data, and links self-reported participant goals with annotated AII, paving the way for AI systems that can better understand and navigate complex social dynamics.

Why it matters

For professionals building AI for social robotics, human-computer interaction, or advanced analytics of group dynamics, COSI-Lab provides an invaluable resource for developing systems that can interpret and respond to nuanced, multi-perspective social intentions.

How to implement this in your domain

  1. 1Utilize the COSI-Lab dataset to train and evaluate AI models for understanding multi-perspective social intentions in complex group settings.
  2. 2Develop AI systems that can infer "Apparent Intent Inference" by considering diverse observer perspectives, rather than a single objective truth.
  3. 3Integrate multimodal data streams (audio, visual, sensor) to enhance AI's ability to interpret nonverbal social cues and intentions.
  4. 4Design AI applications that can adapt their behavior based on inferred social intentions, improving human-AI collaboration in dynamic environments.

Original post by Zonghuan Li, Litian Li, Arthur Mercier, Gara Dorta, Balint Dioszegi, Jose Morales-Vargas, Chenxu Hao, Ivan Kondyurin, Vanessa Begemann, Nale Lehmann-Willenbrock, Bernd Dudzik, Saunaq Chakrabarty, Sotiris Vacanas, Laura Cabrera-Quir\'os, Anne L. J. ter Wal, Vitaliy Popov, Jorge Castro-God\'inez, Chirag Raman, Stephanie Tan, Hayley Hung

"arXiv:2607.28649v1 Announce Type: cross Abstract: COSI-Lab presents a multimodal, multi-sensor dataset of an interdisciplinary scientific workshop containing 32 academics at an international conference. It captures ecologically valid social interactions in a weakly scripted setti…"

View on X

Originally posted by Zonghuan Li, Litian Li, Arthur Mercier, Gara Dorta, Balint Dioszegi, Jose Morales-Vargas, Chenxu Hao, Ivan Kondyurin, Vanessa Begemann, Nale Lehmann-Willenbrock, Bernd Dudzik, Saunaq Chakrabarty, Sotiris Vacanas, Laura Cabrera-Quir\'os, Anne L. J. ter Wal, Vitaliy Popov, Jorge Castro-God\'inez, Chirag Raman, Stephanie Tan, Hayley Hung on X · view source

Want to go deeper?

Turn these trends into skills with Learnijoy's hands-on AI & tech courses.

Explore courses