CJSD Discriminators Isolate Data Shift Types for ML.

Machine learning models operating on continuous data streams frequently encounter concept drift, which can manifest as changes in input data distribution (covariate shift) or alterations in the underlying relationship between inputs and outputs (mechanism change). Accurately identifying the type of drift is crucial for maintaining model performance and efficiently managing model lifecycle. This paper presents CJSD, a novel approach that employs two discriminators to measure a conditional discrepancy, enabling an exact decomposition of these two types of data shifts. This statistical method provides a robust foundation for decision-making in systems that manage pools of expert models. The framework allows for statistically meaningful decisions to reuse an existing model, spawn a new one, or defer action, all backed by finite-time, anytime validity guarantees. It also incorporates a restarted e-detector bank to efficiently handle recency and control error rates across an unbounded number of experts, ensuring the system remains adaptive and reliable.