AI Improves Simulation Model Discovery with Natural Language Queries

Discovering and reusing existing simulation models presents a significant challenge in Modeling and Simulation (M&S), especially when dealing with extensive model repositories. Identifying models that precisely match a specific modeling intent can be difficult. Recent advancements in Artificial Intelligence, particularly retrieval-based methods, offer a promising solution by operating at a semantic level to bridge this gap. This paper details an experimental study investigating how various factors influence the effectiveness of AI-driven simulation model discovery using natural language queries. The research examined the impact of data representation, the choice of transformer-based embedding models, and different retrieval strategies. Key findings indicate that the way data is structured significantly affects performance, open-source embedding models can achieve high accuracy, and reranking methods are crucial, especially as query complexity increases. This work establishes a foundational benchmark for AI-driven model discovery, contributing to the broader goal of achieving AI-driven composability and interoperability in M&S.