Learnijoy
ResearchAI Engineering & DevToolsAI Research

ZeroBN Optimizes DNNs for Edge Latency Constraints

Shuo Huai, Di Liu, Hao Kong, Weichen Liu, Ravi Subramaniam, Christian Makaya, Qian Lin· July 9, 2026 View original

▶ The 2-minute explainer

Summary

This paper introduces a latency-oriented neural network learning method using Zerorized Batch Normalization (ZeroBN) to optimize DNNs for edge devices. It achieves high accuracy while strictly adhering to latency constraints, demonstrating significant improvements over state-of-the-art methods on various models and hardware.

Deploying deep learning applications on edge devices is crucial for addressing privacy and latency concerns associated with cloud servers. However, designing deep neural networks to maximize performance while meeting strict real-time latency requirements on edge systems remains a challenge, as conventional optimization methods don't directly control inference time. This research proposes a novel latency-oriented neural network learning method that utilizes "Zerorized Batch Normalization" (ZeroBN). The goal is to optimize models for high accuracy while strictly adhering to predefined latency constraints. A key innovation is a universal, hardware-customized latency predictor that enables this optimization through a single training process. Experiments show that ZeroBN effectively meets hard latency constraints and maintains high accuracy. For example, it reduced GoogLeNet's latency on an NVIDIA Jetson Nano from 40.32 ms to 34 ms with only a minor accuracy drop, and even improved VGG-19's accuracy on a Jetson TX2 while significantly reducing its latency.

Why it matters

AI engineers and product developers working on edge computing can use this method to deploy high-performing deep neural networks that reliably meet real-time latency requirements, crucial for applications like autonomous vehicles, industrial IoT, and smart cameras.

How to implement this in your domain

  1. 1Evaluate existing DNN models deployed on edge devices against their latency constraints.
  2. 2Explore the ZeroBN framework and its latency-oriented optimization techniques.
  3. 3Implement and benchmark ZeroBN on target edge hardware (e.g., NVIDIA Jetson series) for specific applications.
  4. 4Integrate the hardware-customized latency predictor into the model design and training pipeline.

Who benefits

AutomotiveIndustrial IoTRoboticsSmart DevicesTelecommunications

Key takeaways

  • ZeroBN optimizes DNN architectures for edge systems with strict latency constraints.
  • It uses a hardware-customized latency predictor for efficient one-shot training.
  • The method achieves high accuracy while strictly meeting latency targets.
  • Significant improvements in latency and sometimes accuracy are observed on edge hardware.

Original post by Shuo Huai, Di Liu, Hao Kong, Weichen Liu, Ravi Subramaniam, Christian Makaya, Qian Lin

"arXiv:2607.06922v1 Announce Type: new Abstract: Deep learning applications have been widely adopted on edge devices, to mitigate the privacy and latency issues of accessing cloud servers. Deciding the number of neurons during the design of a deep neural network to maximize perfor…"

View on X

Primary sources

Originally posted by Shuo Huai, Di Liu, Hao Kong, Weichen Liu, Ravi Subramaniam, Christian Makaya, Qian Lin on X · view source

Want to go deeper?

Turn these trends into skills with Learnijoy's hands-on AI & tech courses.

Explore courses

More in AI Engineering & DevTools

AI ResearchAI Engineering & DevTools

Transformers Learn Non-Invertible Modular Multiplication via Stratified Fourier Mechanisms.

This research investigates how small transformers learn modular integer multiplication over composite moduli, a non-invertible operation. It proposes the "monoid extension" theory, suggesting models partition input space into hierarchical algebraic regions where Fourier mechanisms apply, explaining how embeddings, attention, and local features contribute to the computation.

Zitong Andrew Chen, Junaid Hasan, Akhil Srinivasan, Hemkesh Bandi, Jarod AlperJul 9, 2026
AI Engineering & DevToolsAI Research

New Interpretable Model Handles Feature Interactions in Tabular Data.

This paper introduces Interaction Aware Interpretable Machine Learning (IAIML), a framework for tabular data that addresses the limitation of traditional interpretable models in capturing feature interactions. IAIML uses adaptive discretization, pairwise interaction scoring, and a partitioned explanation budget to achieve high accuracy while maintaining interpretability.

Srikumar KrishnamoorthyJul 9, 2026
AI ResearchAI Engineering & DevTools

Principles of Deep Feedforward ReLU Networks Unveiled.

This paper systematically studies the mechanisms of deep feedforward ReLU networks, generalizing principles from two-layer networks to deeper architectures. It explains how hidden-layer units form piecewise linear manifolds to divide input space and how paths and their relationships are central to understanding the back-propagation training solution.

Changcun HuangJul 9, 2026