New Framework Detects MLaaS Performance Drift in IoT

This research addresses the critical challenge of performance drift in Machine Learning as a Service (MLaaS) applications deployed within dynamic Internet of Things (IoT) environments. The constantly changing data distributions in IoT, coupled with periodic MLaaS updates, can degrade model performance. A significant hurdle is that MLaaS clients typically operate as black-box users, lacking access to internal data or parameters, which complicates drift detection. To overcome this, a novel MLaaS Performance Drift Detection framework is introduced. It begins with an MLaaS extraction model that learns the service's behavior from input-output pairs and identifies features influencing predictions. Building on this, the MLaaS Performance Drift Detection (MPDD) model is designed to simultaneously capture variations in both input data and MLaaS behavior. Furthermore, an Adaptive-Temporal Performance Drift Detection Mechanism (APDDM) is proposed. This mechanism dynamically adjusts the monitoring frequency based on observed behavioral and data variations, ensuring timely detection of drift for effective service management. Extensive experiments using real-world datasets demonstrated that MPDD significantly improves accuracy over baseline methods, and APDDM further enhances accuracy while reducing miss detection rates.