LLMs Outperform Supervised Models in Cross-Dataset Bloom Question Classification.

Automatically classifying assessment questions according to Bloom's taxonomy can significantly reduce the workload for instructors, but the process is often subjective and varies between teachers. While previous machine learning and deep learning methods showed strong results within specific datasets, their real-world generalizability across different datasets remained unclear. Furthermore, the effectiveness of large language models (LLMs) for this task had not been systematically studied. This research systematically evaluated both existing ML/DL methods and LLMs with various prompting strategies on five different datasets to assess their cross-dataset generalization. The findings revealed that traditional supervised ML/DL models experienced substantial performance degradation when applied to unseen datasets. In contrast, LLMs demonstrated greater stability and robustness across diverse educational contexts. The most effective prompting strategy involved combining in-context examples with course-specific action verbs. Based on these insights, a lightweight user interface was developed to help instructors automatically classify large question banks, which a usability study confirmed to be highly usable and reduce workload.