Integrated Pipeline Predicts Student Performance, Analyzes Metacognition.

Predicting student performance and understanding their metacognitive calibration—how well they assess their own knowledge—are crucial for developing effective intelligent tutoring systems. Traditionally, these analyses are conducted in separate pipelines, making a unified interpretation challenging. This research introduces the Unified Behavioral Prediction and Calibration Analysis Pipeline (UBP-CAP) to integrate these processes. UBP-CAP processes student behavioral telemetry, such as pre-execution data, through three interconnected modules. First, a LightGBM classifier, augmented with SHAP for interpretability, predicts binary correctness. Second, formal calibration metrics like Expected Calibration Error (ECE) and Brier score decomposition evaluate the alignment between student confidence and actual performance. Third, a crossed Generalized Linear Mixed-Effects Model (GLMM) decomposes calibration deviations, identifying factors influencing miscalibration. The pipeline also introduces the Predictive-Explanatory Divergence Index (PEDI), which quantifies the structural difference between predictive and explanatory feature profiles. Evaluated on a dataset of student interactions, Logistic Regression surprisingly achieved higher AUC-ROC (0.903) than LightGBM (0.878). The study found significant student miscalibration (naive ECE = 0.109 vs. model ECE = 0.068) and that calibration is more situational than dispositional (ICCStudent = 0.123). PEDI indicated structural alignment between prediction and explanation on shared features.