Sim2Signal Benchmarks Bridge Sim-to-Real Gap in Traffic Control

This paper introduces Sim2Signal, a novel benchmark aimed at addressing the significant challenge of the "Sim-to-Real" gap in reinforcement learning (RL) applications for traffic signal control. While RL policies often perform well in simulated environments, their effectiveness frequently diminishes when deployed in real-world traffic systems. This discrepancy stems from various sources, including mismatches in sensing, action execution, traffic dynamics, and control objectives. Sim2Signal systematically breaks down this gap into four distinct components: observation, action, transition, and reward gaps, each corresponding to a mismatch in the underlying Markov Decision Process (MDP). The benchmark allows for inducing each gap in isolation under a standardized protocol. Researchers evaluated 18 different mitigation methods on two base controllers across 33 gap settings and 10 calibrated networks derived from five real-world locations. The findings indicate that direct transfer of policies consistently degrades performance across all gap sources. Interestingly, the severity of this degradation does not reliably predict the effectiveness of mitigation strategies. Instead, mitigation success is highly dependent on the specific network and gap setting. The study concludes that the most effective methods generally involve estimating what the gap changes, rather than relying on domain randomization or invariant representations to make policies insensitive.