Reinforcement Learning Cuts AI Datacenter Energy Consumption

Researchers have investigated the power consumption of reinforcement learning post-training for large language models (LLMs) and developed an innovative solution to optimize energy use in datacenters. Current datacenter GPU power management often relies on static caps or reactive throttling, which can indiscriminately slow hardware. This study introduced a PPO meta-controller, trained with reinforcement learning, to dynamically adapt LLM workload generation parameters based on real-time power telemetry. The controller was tested at various scales, from single GPUs to a fleet of 16 A100s. On a 7B model, it cut power-limit violations by nearly 90% while simultaneously increasing token output by over 18% and energy efficiency by 26%. While initial deployment at 72B scale showed null results due to actuator authority issues under model sharding, a re-engineered controller using generation concurrency as an actuator successfully managed a live 72B rollout, achieving 35.7% more output than a static baseline with minimal budget violations. The findings suggest that dynamic, RL-based power control can lead to significant economic and carbon savings. For a 16-GPU fleet, peak demand was reduced to 50-56% of nameplate, indicating that roughly twofold oversubscription of nameplate power capacity could be feasible. This research provides a practical pathway for datacenters to optimize energy consumption and improve throughput for LLM training and inference.