New Framework Defines Epistemic Trustworthiness for Generative AI

Nimisha Karnatak, Max Van Kleek, Nigel Shadbolt· August 7, 2026 View original

Key takeaways

  • Epistemic trustworthiness defines when reliance on GenAI outputs is justified, beyond mere accuracy.
  • It comprises three conditions: epistemic humility, access, and resistance to injustice.
  • Failures in these conditions can lead to significant harms in high-stakes applications.
  • The framework guides the design and evaluation of GenAI for warranted professional reliance.

Who benefits

HealthcareLegalFinancial ServicesEducationGovernment

Summary

This paper introduces a normative framework for "epistemic trustworthiness" in generative AI, focusing on conditions under which users are justified in relying on AI outputs in high-stakes professional contexts. It proposes three conditions: epistemic humility, epistemic access, and resistance to epistemic injustice.

As generative AI systems become integral to high-stakes professional workflows, the question of when users can genuinely trust and rely on their outputs becomes paramount. Existing responsible AI frameworks often focus on accuracy, fairness, or explainability, but these do not directly address the conditions for "epistemically warranted reliance"—that is, when users are justified in incorporating AI outputs into their own reasoning processes. This research proposes a new normative framework centered on "epistemic trustworthiness," defining what makes an AI system worthy of intellectual reliance. Drawing from philosophical concepts of trustworthiness, the framework outlines three jointly necessary and non-fungible conditions. First, "epistemic humility" requires systems to clearly communicate their competence limits. Second, "epistemic access" demands that systems allow users to inspect, question, and contest outputs within their context. Third, "resistance to epistemic injustice" mandates that systems recognize users as legitimate knowledge agents and avoid marginalizing their expertise. Through case studies in legal, medical, and hiring domains, the paper illustrates how failures in these three conditions can lead to significant harms that go unaddressed by traditional metrics like accuracy or fairness alone. The framework provides a crucial lens for designing and evaluating GenAI systems to foster justified reliance rather than merely correct outputs.

Why it matters

For professionals deploying or relying on generative AI in critical applications, understanding epistemic trustworthiness is essential for mitigating risks, ensuring ethical use, and building systems that users can genuinely trust, not just use.

How to implement this in your domain

  1. 1Integrate principles of epistemic humility into AI system design by clearly communicating confidence scores, uncertainty ranges, and potential limitations of outputs.
  2. 2Develop user interfaces and tools that provide epistemic access, allowing users to trace AI reasoning, inspect source data, and easily provide feedback or corrections.
  3. 3Implement mechanisms to resist epistemic injustice by ensuring diverse user perspectives are considered in AI development and by empowering users to challenge AI outputs without bias.
  4. 4Conduct internal audits of high-stakes GenAI applications against this framework to identify and address potential trustworthiness gaps.

Original post by Nimisha Karnatak, Max Van Kleek, Nigel Shadbolt

"arXiv:2608.05602v1 Announce Type: new Abstract: Generative AI systems are increasingly deployed in high-stakes professional contexts, where their outputs shape what users believe, how they reason, and what they treat as settled. This raises a central question for responsible AI:…"

View on X

Originally posted by Nimisha Karnatak, Max Van Kleek, Nigel Shadbolt on X · view source

Want to go deeper?

Turn these trends into skills with Learnijoy's hands-on AI & tech courses.

Explore courses