Offline RL Controls Fluids with Adaptive Sensor Policies

Active flow control, a critical engineering application, has seen advancements with deep reinforcement learning (RL). However, traditional online RL methods demand extensive real-time interaction with high-fidelity environments and require complete policy retraining for every sensor configuration change, making them computationally prohibitive for real-world deployment. This new research introduces an innovative offline RL framework designed to overcome these limitations through data-driven policy extraction. A key feature is its sensor position-conditioned architecture, which incorporates Point Attention layers to model spatial relationships. This design enables a single policy network to seamlessly adapt to diverse sensor arrangements without the need for retraining. The framework's effectiveness was demonstrated on two complex problems: mitigating chaotic behavior in the Kuramoto-Sivashinsky equation and controlling flow over airfoils governed by Navier-Stokes equations. The results highlight the unprecedented flexibility this approach offers for optimizing sensor placement, marking a significant step towards developing adaptive and intelligent flow control systems.