Study Strategies Predict Engagement, Not Learning Mastery, in EdTech

In learning analytics, it's common to interpret unsupervised clusters derived from intelligent tutoring system (ITS) logs as distinct learner types that should correlate with learning outcomes. This research investigates that assumption using the EdNet-KT3 dataset. By clustering study-strategy features—such as resource use, revision habits, video consumption, and problem practice—for 5,000 active learners, the study identified eight distinct study styles. These clusters were then used to predict later learner outcomes, splitting each learner's timeline to ensure early behaviors predicted late outcomes. The findings revealed that early study-strategy clusters were strong predictors of later engagement, including persistence in practice and session completion. However, these clusters did not significantly predict later unassisted accuracy (correctness on first attempts without help) or knowledge gains. Furthermore, a knowledge-tracing model (SAKT) showed only modest improvement over a baseline in predicting correctness, and this mastery signal was largely independent of the observed behavioral styles. This suggests that while behavioral patterns in ITS logs can effectively describe how students engage with learning materials, they do not necessarily reflect their actual knowledge acquisition or mastery.