Optimizing Memory and Compute for Energy-Efficient DNN Inference

Deep Neural Network (DNN) inference on mobile devices often faces challenges with high latency and energy consumption due to limited hardware resources. While many existing studies focus on dynamic voltage and frequency scaling (DVFS) for computing frequency, the impact of memory frequency on inference performance has largely been overlooked. This research addresses this gap by jointly optimizing both memory and computing frequencies, along with communication resources, for energy-efficient DNN inference.The authors developed a realistic inference time model and formulated an optimization problem to minimize the total energy consumption across mobile devices, all while adhering to strict deadline constraints. For local inference scenarios, they derived a near-optimal closed-form solution using convex optimization. For edge inference, an optimal closed-form solution for transmission power was obtained given a fixed bandwidth.Furthermore, a low-complexity heuristic algorithm was proposed to effectively solve the overall problem with polynomial time complexity. Simulation results, based on measured data, demonstrated that the proposed near-optimal solution for local inference achieved optimal performance under strict deadlines, with a minimal performance gap. Crucially, the algorithm significantly reduced device energy consumption by up to 10.4% compared to other methods, highlighting the importance of joint optimization.