KGRL Boosts Reinforcement Learning with Knowledge and Gradients.

Reinforcement Learning (RL) in Parametrized Action Markov Decision Processes (PAMDPs) often struggles with sample inefficiency because it typically relies on one-shot estimators for numerical parameters. While explicit domain knowledge, such as rules or safety constraints, is often available in PAMDP environments, it is rarely directly utilized to enhance the training efficiency of RL agents. This research bridges that gap by proposing the Neuro-Symbolic Knowledge- and Gradient-Guided Reinforcement Learning (KGRL) algorithm. KGRL leverages domain knowledge stored in a Datalog knowledge base to intelligently prune the decision space by identifying and removing non-applicable actions and constraining the parameter spaces of the remaining actions for any given state. Following this knowledge-guided pruning, KGRL employs a gradient-based parameter refinement loop to accurately estimate optimal parameters during both training and deployment. This dual approach significantly guides the agent's exploration towards feasible and constraint-aware decisions, leading to increased sample efficiency. The algorithm also provides local procedural explanations by recording activated rules, and it demonstrates superior performance in both sample efficiency and episodic return compared to existing RL baselines for PAMDPs.