InCarEmo Dataset Boosts Driver Emotion and State Monitoring

Understanding a driver's emotions and overall state is crucial for developing the next generation of intelligent in-cabin systems, which aim to enhance safety and improve human-vehicle interaction. However, existing public datasets for in-cabin affective computing are often limited to visual modalities and lack conversational data, making it difficult to capture the full range of linguistic and interactive cues that underpin driver emotion. To address these limitations, researchers introduce InCarEmo, a comprehensive multimodal dataset. It combines RGB and infrared video, in-cabin audio, and dialogue text, all collected from carefully scripted in-cabin scenarios designed to simulate realistic driver behaviors under diverse lighting conditions and driving contexts. The dataset supports three key tasks: multimodal emotion recognition, fatigue detection, and distraction monitoring. In addition to the original Chinese data, an auxiliary English benchmark is provided for cross-lingual evaluation. Extensive baseline results using unimodal and multimodal methods, including analyses under noisy and low-light conditions, demonstrate the benefits of multimodal fusion while highlighting remaining challenges. InCarEmo aims to establish a robust foundation for interpretable and human-centric in-cabin affective understanding.