AI Optimizes Geothermal Well Control with Diffusion-Surrogate RL

Real-time decision-making in enhanced geothermal systems (EGS) presents significant challenges due to long production periods, high-dimensional control spaces, and the computational expense of high-fidelity hydrothermal simulations. To address this, a new framework called diffusion-surrogate guided reinforcement learning has been introduced for optimizing EGS well control. This approach leverages reinforcement learning (RL) for state-dependent sequential control, but bypasses the costly direct training with numerical simulators. The core of the framework involves constructing a learned surrogate environment using conditional diffusion models. This surrogate accurately predicts the evolution of reservoir temperature and pressure fields, which serve as system states, while injection rates are chosen as control actions. A separate reward model estimates the economic return associated with these actions. This surrogate environment is then integrated with Proximal Policy Optimization (PPO) for efficient policy training. Experiments conducted on a fractured EGS benchmark demonstrate that the diffusion surrogate can accurately reproduce reservoir-state evolution over multiple control stages. The resulting surrogate-assisted PPO policy achieves performance comparable to direct simulator-based PPO and other existing optimization methods, but with a substantial reduction in reliance on expensive high-fidelity simulations. This highlights the potential of diffusion-based surrogate environments for efficient RL in complex, computationally intensive optimization problems like geothermal well control.