Self-Play Driving Policies Achieve Human-Like Autonomy with Minimal Data

Self-play reinforcement learning has emerged as a promising technique for training autonomous driving policies, leveraging large-scale simulations to reduce reliance on expensive human driving demonstrations. However, a key challenge with purely self-play trained policies is their tendency to develop effective but unconventional driving behaviors that are incompatible with human expectations. Traditional attempts to align these behaviors with human norms often involve extensive and labor-intensive reward engineering or domain randomization. This new research proposes a more efficient solution: treating a minimal amount of human driving data as a regularization objective, layered on top of a basic safe goal-reaching reward. The findings indicate that even a small quantity of human data—as little as 30 minutes, which is 2500 times less than comparable imitation learning methods—can significantly influence the outcome. The resulting policies demonstrate coordination with human trajectories and can be trained efficiently on consumer-grade GPUs, suggesting a scalable path to more human-compatible autonomous systems.