Multimodal AI for Searchable Aerial Imagery at Scale

This article explores the development of a scalable multimodal AI system designed for searching vast amounts of aerial imagery. It outlines the architectural choices, leveraging AWS services like Amazon Bedrock and Amazon OpenSearch Serverless, to create a robust solution for geospatial semantic search. The authors detail their evaluation methodology, which uses OpenStreetMap as ground truth, and present findings from four experiments comparing various embedding models, fusion techniques, captioning methods, and search strategies. The post offers practical advice for professionals looking to build similar systems, highlighting key design decisions that significantly impact performance. Specifically, it identifies Amazon Nova Multimodal Embeddings as a top performer, achieving the highest F1 scores in their benchmark queries. This work has been commercialized into Vexcel Intelligence, a product for searchable imagery.