SmartRAG Brings Graph-Based AI Assistants to Mobile Devices

Deploying powerful large language models (LLMs) as personal assistants directly on mobile devices presents a significant challenge. The need for privacy, low latency, and offline functionality clashes with the high computational demands of large models and the limited hardware resources of edge devices. Traditional model compression alone cannot fully resolve this tension. Researchers introduce SmartRAG, a novel, fully on-device framework that tackles this by decomposing mobile intelligence into four coordinated modules: Perception, Memory, Focus, and Thinking. A key component is EvoNER, a named-entity recognizer that continuously learns and expands its label inventory through teacher-distilled updates, allowing the system to absorb new entity types without retraining the core LLM. Extracted knowledge is stored in MRGraph, a three-layer knowledge graph that preserves provenance. This knowledge is retrieved via a hybrid pipeline combining graph traversal, lexical matching, and semantic search. The on-device LLM is strategically invoked only for high-value tasks like labeling, planning, and answer synthesis, keeping inference costs manageable. SmartRAG, using a quantized 1.7B-parameter LLM, demonstrates multi-hop reasoning performance comparable to models 18 times larger, all while operating efficiently on standard smartphones.