Optimizing Foundation Model Deployment for Transportation Management Centers

Transportation Management Centers (TMCs) are increasingly adopting foundation models, including large language models (LLMs) and vision-language models (VLMs), for tasks like anomaly detection, incident reporting, and traveler information. However, deciding which models to deploy for specific functions, in what mode (e.g., open-source API, closed API, on-premise), and under a shared hardware budget presents a complex optimization challenge. Researchers have formulated this as the Foundation Model Deployment Portfolio (FMDP) problem. This is a mixed-integer program designed to minimize the total cost of ownership (TCO) while adhering to critical constraints such as per-function quality, latency, safety requirements, and available shared GPU capacity. The problem is proven to be NP-hard, and a polynomial-time greedy heuristic is proposed for practical solutions. An illustrative case study involving five TMC functions and 19 candidate model-mode pairs demonstrated FMDP's effectiveness. It identified a mixed deployment portfolio costing significantly less than an all-closed-API baseline, primarily by routing most functions to open-source APIs and reserving closed APIs only for functions with stringent quality floors. The analysis also provided insights into the break-even point for on-premise GPU investments, indicating that such investments become viable only at high query volumes or if API prices substantially increase.