CineCap: Structured Reasoning for Cinematographic Video Captioning

Cinematographic captioning, which involves describing how a video is filmed using professional film language (e.g., camera movement, shot size, depth of field), is crucial for fine-grained video understanding and controllable video generation. Despite its importance, this capability remains largely unexplored in current multimodal large language models. The task is challenging because it requires inferring subtle professional concepts and generating both comprehensive and accurate open-form descriptions across multiple cinematic dimensions. To address these challenges, researchers propose CineCap, a novel framework that combines structured reasoning with spatio-temporal anchors and reinforcement learning. The structured reasoning component grounds professional cinematographic descriptions in explicit visual evidence, organizing them into compact atomic reasoning units for supervised fine-tuning. The reinforcement learning aspect, utilizing comprehensiveness, accuracy, and gated coverage rewards, helps balance descriptive completeness with factual correctness. The team also constructed CineCap Bench, a new benchmark of 472 manually annotated video-caption pairs for systematic evaluation. Extensive experiments show CineCap consistently outperforms strong baselines, setting a new state of the art.