MiCoPro Optimizes Mixed-Precision AI for Edge Devices.

Quantized Neural Networks (QNNs) using low-bitwidth data are crucial for efficient storage and computation on edge devices. Mixed-precision quantization (MPQ), which applies different bitwidths to different layers, is a popular strategy to balance accuracy and speed. However, existing MPQ exploration algorithms often lack flexibility and efficiency, struggling to predict the complex impact of various MPQ schemes on model performance. Furthermore, a comprehensive end-to-end framework for MPQ optimization and deployment has been missing. To address these challenges, researchers developed MiCo, a holistic framework for MPQ exploration and deployment in edge AI. MiCo incorporates a novel optimization algorithm designed to find accuracy-optimal quantization configurations while adhering to strict latency constraints. This framework was further extended to MiCoPro, which introduces a robust Hardware-Aware Proxy (HAP) model. The HAP model significantly enhances prediction accuracy and hardware versatility by leveraging target-specific latency modeling. MiCoPro enables rapid exploration of MPQ schemes and direct deployment from PyTorch models to bare-metal C code. Demonstrations on both the BitFusion accelerator and SIMD-extended RISC-V processors show that MiCoPro can achieve up to 40% latency reduction with less than a 3% accuracy drop, making it a powerful tool for efficient edge AI.