CRAWO Orchestrates AI Workloads on Heterogeneous Edge Devices

Edge intelligence is crucial for real-time smart city applications, moving computation closer to data sources to reduce latency and bandwidth. However, deploying AI pipelines across varied edge hardware, from microcontrollers to accelerator-equipped systems, presents significant challenges. Existing orchestration platforms often fall short in adaptively allocating resources under dynamic conditions. This paper introduces CRAWO, a novel architectural framework for orchestrating AI workloads in distributed edge environments. CRAWO employs a control-loop mechanism that separates allocation logic from execution, handling placement decisions, state management, and inter-stage data flows. It instantiates services directly on edge nodes and includes a hardware-aware allocator with a pluggable multi-criteria decision layer, utilizing real-time infrastructure metrics for intelligent workload placement. The reference implementation uses a microservices architecture on K3s, leveraging Kubernetes Custom Resource Definitions (CRDs) and a dedicated operator. Evaluation in a vehicle surveillance scenario, specifically license plate recognition, demonstrated that CRAWO improves workload distribution and reduces reliance on centralized cloud processing, which is critical for latency-sensitive applications.