Improving Trajectory Forecasting by Aligning Training and Inference

Trajectory forecasting, particularly for autonomous driving, has seen rapid advancements. However, a common issue is that many models produce posterior probabilities over forecast modes that are not very informative, complicating the process of selecting the most likely trajectory. This research attributes this problem to a fundamental mismatch between how these forecasters are modeled and how they are trained. Typically, forecasters are conceptualized as conditional Gaussian Mixture Models (GMMs), but they are often trained using a winner-take-all (WTA) loss function. This WTA approach assigns each training sample exclusively to its closest mode, akin to K-means clustering. While this hard assignment helps prevent mode collapse, the authors argue it leads to uninformative mode probabilities because it over-segments the trajectory space, disregards the relationships between nearby modes, and introduces instability in assignments with minor input perturbations. To rectify this, the paper introduces two lightweight, post-hoc solutions that do not require retraining the model. The first is a test-time posterior-weighted merging technique that aggregates nearby candidate trajectories. The second is a one-step Expectation-Maximization (EM) update, which replaces the hard assignments with soft responsibilities, allowing probability mass to be shared across neighboring modes. These methods consistently yield more informative and accurately ranked mode posteriors, improving final forecasts on standard displacement metrics across various WTA-trained architectures. This work unifies previous design choices through a GMM-vs-K-means lens, offering practical and principled corrections.