New Causal Abstraction Method Improves MDP Scalability and Policy Learning

Markov Decision Processes (MDPs) are fundamental models for decision-making, but their complexity grows exponentially with state space size, making many reasoning tasks difficult. This new work proposes a method to simplify MDPs by creating "causal abstractions." These abstractions focus on the underlying causal relationships between state variables and their properties. The technique identifies groups of states that exhibit similar causal patterns regarding specific properties, effectively reducing the overall complexity of the MDP. By doing so, the abstracted model retains crucial characteristics of the original, allowing for more efficient analysis. Evaluations show that this approach generates smaller, more manageable MDPs, from which near-optimal policies for the original, larger systems can be derived. The method also demonstrates generalizability across various MDP models and benchmarks.