KGCache Accelerates Knowledge Graph Reasoning for LLMs

Large Language Models (LLMs) can provide more accurate answers to knowledge-intensive questions when grounded with Knowledge Graphs (KGs). However, existing KGQA systems often repeatedly query the same parts of the knowledge graph, leading to inefficiencies. KGCache is a new in-memory caching solution designed to address this by storing frequently accessed KG neighborhoods. KGCache is positioned between the KGQA engine and the backend KG, allowing it to intercept and serve repeated entity requests directly from its cache. This design is compatible with both iterative traversal and one-shot planning KGQA paradigms, making it versatile. The system aims to reduce the latency associated with fetching information from the knowledge graph. Evaluations on benchmarks like WebQSP and CWQ demonstrated that KGCache significantly accelerates KG retrieval, achieving up to a 1.91x speedup. Semantic caching for similar queries further boosted hit rates, leading to full-system speedups. This innovation makes KG-grounded LLMs more efficient, enabling faster and potentially more scalable knowledge-intensive applications.