New AutoThread Method Boosts RL Inference Speed by Mitigating Simulation Bottlenecks

Reinforcement Learning (RL) systems that rely on simulations often face performance bottlenecks due to the simulator's execution overhead, especially with highly dynamic workloads. Current multithreaded approaches struggle to adapt thread resources effectively, leading to inefficiencies like resource contention and reduced throughput. This new research identifies the ratio of task execution time to scheduling time as a critical factor for optimal thread allocation. Building on this insight, the proposed AutoThread method offers a hybrid adaptive solution. It leverages a Physics-Informed Neural Operator (PINO) to predict optimal thread counts, guided and constrained by a finite-source M/M/1 queueing model for fast and accurate estimation under varying loads. Additionally, AutoThread includes load-aware online fine-tuning to correct prediction errors and refine resource allocation in real-time. Experimental results demonstrate that AutoThread significantly enhances performance, achieving an average speedup of 18.4% compared to static strategies. It also delivers 1.7x and 1.8x higher throughput than XGBoost and Reinforcer, respectively, and can reduce execution time by up to 83.8% against state-of-the-art methods, making it highly effective for high-performance RL inference.