EvoLP Predicts Edge AI Latency, Guides Model Compression

Deploying deep learning applications on resource-constrained edge devices requires careful consideration of real-time performance and latency. Accurately measuring inference latency on physical edge hardware is often challenging and expensive, hindering efficient model compression efforts. Researchers have developed EvoLP, a novel and efficient framework designed to predict the inference latency of models on edge devices with high accuracy. A key feature of EvoLP is its ability to self-evolve, continuously improving its prediction precision throughout the network compression process. Experimental evaluations across three different edge devices and four model variants demonstrate that EvoLP surpasses existing state-of-the-art latency prediction methods. When integrated into a model compression framework, EvoLP effectively directs the compression to achieve superior model accuracy while strictly adhering to predefined latency constraints. The framework is open-sourced for broader adoption.