New AI Framework Enhances Air Traffic Controller Situational Awareness

As air traffic volumes continue to grow, particularly in lower altitude airspaces, there's an increasing need for advanced decision support tools for air traffic controllers. This paper presents V2TATC (Voice-to-Trajectory for Air Traffic Control), an innovative framework that combines voice communication data with flight trajectory information. The core idea behind V2TATC is to create a joint embedding space where both spoken instructions from pilots and the real-time flight paths of aircraft are represented. This allows the system to understand the relationship between what is said and what is happening physically, recognizing that these distinct data types refer to the same aircraft and its actions. The framework integrates a self-supervised trajectory encoder, a pre-trained speech encoder, a contrastive joint embedding module, and normalizing flows for bijective lifting. Tested effectively in the complex San Francisco Bay Area airspace, V2TATC facilitates cross-modal retrieval and offers a novel paired voice-trajectory dataset for further research, promising enhanced situational awareness for controllers.