Interpreting Black-Box LLMs Using Sentence-Level Energy Landscapes

The widespread use of proprietary Large Language Models (LLMs) via closed APIs presents a significant challenge for interpretability, which is crucial for responsible AI deployment. To address this, researchers introduce a novel, model-agnostic, post-hoc attribution interpreter that operates at the sentence level. Their method involves training an Energy-Based Model (EBM) as a surrogate to effectively model the LLM's internal conceptual consistency between input prompts and generated responses. This energy landscape then serves to guide the training of a lightweight interpreter network. A key innovation is that once trained, this interpreter functions independently, quantifying the influence of specific prompt sentences on a user-defined target output without needing additional API queries to the original LLM. By globally training a local interpreter across a diverse range of inputs, the framework aims to capture broader generation patterns and mitigate biases that might arise from instance-specific interpretations. Experiments confirm that the EBM accurately simulates the target LLM, enabling the interpreter to effectively pinpoint the most influential prompt sentences for particular outputs.