Shapley Context Pruning Boosts RAG Efficiency with Game Theory.

Modern RAG systems rely heavily on efficient context reranking and pruning to perform well. Existing methods often use heuristic loss functions and empirical attribution, lacking a unified, interpretable framework. This new research proposes Shapley Context Pruning (SCP), which frames context reranking through the lens of cooperative game theory. SCP models the context as a cooperative game, allowing for a more principled approach to attributing importance to different parts of the context. It uses a Deep Sets architecture to approximate a permutation-invariant value function at the sentence level, leveraging pre-trained language models for embeddings. The framework employs Monte-Carlo sampling for scalability and provides theoretical guarantees for preserving top-K rankings. Experiments across various tasks, including supporting-sentence recall, Needle-in-the-Haystack evaluations, and long-context QA, demonstrate that SCP achieves competitive performance compared to established baselines, while offering enhanced interpretability through its game-theoretic foundation.