Knowledge Graphs: How to Effectively Guide Reinforcement Learning

This research investigates the conditions under which knowledge graphs (KGs) effectively enhance reinforcement learning (RL) performance. Previous studies often presented KGs as universally beneficial, but this work systematically explores when they help, hinder, or have no impact. By varying RL tasks, KG injection mechanisms (state features, action masking, potential-based reward shaping), and KG quality in MiniGrid environments, the study provides clear insights. Key findings indicate that structured KG guidance improves sample efficiency and reliability in compositional sparse-reward tasks, with benefits collapsing if KG edges are permuted, confirming the structural nature of the gain. The value of a KG directly correlates with the amount of task-relevant knowledge it contains. Crucially, the injection mechanism dictates safety: soft, optimality-preserving methods benefit from correct knowledge and gracefully ignore incorrect information, whereas hard masking is brittle, potentially forbidding essential actions if the KG is incomplete or flawed. A clinical case study on sepsis management further highlighted that benefits require task structure that the chosen mechanism can exploit.