Multimodal AI Builds Knowledge Graphs from Educational Lectures.

This paper introduces an innovative, evidence-grounded multimodal pipeline designed to construct knowledge graphs from educational lecture videos. Traditional methods often rely solely on transcripts, which fail to capture the rich knowledge distributed across visual elements like slide text, diagrams, equations, and the overall presentation flow. The proposed pipeline meticulously transcribes lectures, identifies semantic anchors, applies optical character recognition (OCR), and leverages a vision-language model. This integrated approach ensures that only concepts and typed relationships explicitly supported by transcript, OCR, or visual evidence are extracted and validated. Mentions are then canonicalized into a provenance-rich knowledge graph. Tested on neural-network lectures, the system successfully processed thousands of frames and segments, yielding a substantial number of canonical concepts and relationships with high endpoint coverage. Preliminary retrieval tests demonstrated perfect accuracy, highlighting the method's potential for enhancing knowledge organization and retrieval in educational contexts.