Curriculum-Grounded LLM-as-Judge Pipeline Enhances Automated Exam Marking

The application of generative AI and large language models (LLMs) in education, particularly for automated assessment, is gaining traction. However, for high-stakes exams, simply using prompt engineering is insufficient; a robust software pipeline is needed to ensure LLM outputs are systematically aligned with authorized curriculum materials and official marking guidelines. A new "LLM-as-Judge" pipeline has been developed, specifically designed for question-level marking in university admission exam preparation. This configurable system identifies relevant topics, subtopics, and cognitive demands of a question, then compiles verifiable context from syllabus artifacts, including prescribed verbs, outcomes, performance descriptors, and glossary definitions. This ensures the LLM's judgment is deeply rooted in educational authority. The pipeline employs a staged LLM workflow. First, it generates question-specific rubrics that outline structured performance expectations. Then, it derives and applies marking criteria to allocate marks to student responses. This design significantly enhances the consistency and transparency of automated marking, ensuring alignment with official practices. Preliminary evaluations indicate that this LLM-as-Judge pipeline produces marking outcomes comparable to human tutors, with justifications that are more traceable to official curriculum and marking standards. Its integration into an online study platform is already providing initial operational insights.