Pruning Framework Optimizes LLMs for Edge Device Deployment

Deploying Large Language Models (LLMs) on embedded and edge computing environments presents significant challenges due to their large parameter counts and high computational demands, which clash with strict latency, memory, and energy constraints. While model pruning can reduce scale, jointly optimizing various architectural components like layers, attention heads, and MLP dimensions is complex and often leads to suboptimal configurations. To address this, researchers propose a hardware-aware, multi-objective structured pruning framework. This two-stage method explicitly targets latency and model size for efficient edge deployment. The initial coarse-grained stage involves depth pruning, removing entire attention and MLP blocks to reduce computational load and memory usage. The subsequent fine-grained stage employs Parallel Bayesian Optimization (PBO) to identify optimal layer-wise pruning ratios under latency constraints. Importance-based strategies then rank specific components for pruning within each layer's allocated budget. Experimental results across multiple LLMs, at 37.5% and 50% pruning ratios, demonstrate that this approach achieves better performance on commonsense reasoning tasks than existing methods while significantly reducing inference cost, making LLMs more viable for edge applications.