HypEMBER Enhances Robust Reinforcement Learning for Dynamical Systems

This work introduces HypEMBER, a new reinforcement learning (RL) framework designed for robust control of parametrized dynamical systems, particularly in the presence of measurement and model uncertainties. Traditional RL methods often struggle with high-dimensional state spaces, expensive solvers, partial knowledge of equations, and parameter uncertainties, leading to poor generalization and lack of robustness. HypEMBER addresses these challenges by employing hypernetworks to represent both policy and value functions. These hypernetworks generate the weights of underlying models conditioned on system parameters, enabling generalization across different dynamic regimes. Additionally, an ensemble of policy and value approximators quantifies epistemic uncertainty, which in turn improves exploration strategies and enhances robustness during and after training. Evaluated on problems like the Kuramoto-Sivashinsky equation and particle navigation, HypEMBER consistently demonstrated superior training stability, sample efficiency, and robustness to noise and parameter misspecification compared to existing state-of-the-art RL methods.