Defining Good Explanations and Challenges for LLM Outputs

The concept of what constitutes a "good explanation" has been a long-standing philosophical debate, which has gained renewed importance with the rise of AI, especially in the context of explaining AI system outputs. The ability to provide clear and understandable explanations is vital for the broader adoption of AI technologies across various sectors. This research introduces a definition for good explanations, drawing inspiration from counterfactual explanations. A key aspect of this definition is the necessity to account for the prior beliefs of the person receiving the explanation. This means an effective explanation must not only address "what if" scenarios but also align with or appropriately challenge the audience's existing knowledge. The paper delves into the implications of this refined definition for AI explainability, particularly focusing on the difficulties encountered when attempting to generate good explanations for outputs from Large Language Models (LLMs). The inherent complexity and black-box nature of LLMs, combined with the need to tailor explanations to individual prior beliefs, make this a significant challenge.