New Affordable Platform Benchmarks AIoT Sim-to-Real Reinforcement Learning Gap

Reinforcement Learning (RL) is crucial for autonomous systems like the AIoT, but training in real-world environments is often expensive and risky. Consequently, most RL research relies on simulations, which introduces challenges when transferring these models to physical systems, a problem known as the Sim-to-Real gap. Evaluating this gap is essential for improving real-world RL performance. A new research paper introduces an affordable, real-world AIoT platform designed specifically to benchmark this Sim-to-Real transferability. The platform, built with commercially available components costing less than $400, allows an edge device agent to play video games on a host computer using hardware-emulated input and vision. This setup minimizes safety risks while providing a tangible real-world environment. Initial experiments revealed a substantial Sim-to-Real gap, with simulation-trained agents experiencing a 1160% performance drop compared to human-level performance upon real-world deployment. However, direct real-world training using a Deep Q-Network (DQN) achieved about 38% of human-level performance, proving the viability of training RL models directly in physical AIoT conditions. This benchmark platform offers a valuable tool for both qualitative and quantitative assessment of RL in real-world AIoT systems.