HYMELL Predicts LLM Latency and Energy Consumption

The increasing scale of Large Language Models (LLMs) has led to significant concerns regarding their computational cost, energy consumption, and inference latency. Accurate estimation of these factors is crucial for sustainable AI deployment and efficient hardware design. This paper introduces HYMELL (Hybrid Modeling for Energy and Latency of LLMs), a novel three-level framework that combines analytical modeling with machine learning to predict LLM inference latency and energy. HYMELL operates hierarchically: it analytically estimates primitive operations, uses machine learning for higher-level components, and an end-to-end model captures system-level overheads across both prefill and decode phases. The framework is versatile, supporting various architectures including dense and Mixture-of-Experts (MoE) feed-forward networks, as well as Multi-Head Attention (MHA) and Grouped-Query Attention (GQA) mechanisms. Evaluated on an NVIDIA H100 GPU, HYMELL demonstrated impressive predictive accuracy, achieving less than 5% error for LLaMA 3 8B in both prefill and decode phases. This capability allows for rapid, hardware-independent design space exploration and optimization for energy efficiency.