AI Advice Reduces Human "I Don't Know" Responses, Despite Errors.

Chiara Marcoccia, Walter Quattrociocchi, Valerio Capraro· July 16, 2026 View original

Key takeaways

  • AI advice significantly reduces human willingness to say "I don't know".
  • Users become less accurate but more confident when using AI, even if wrong.
  • Incentivizing accuracy doesn't fully counteract this over-reliance.
  • AI can alter human metacognition, impacting judgment thresholds.

Who benefits

HealthcareFinanceEducationLegalConsulting

Summary

A study found that access to AI advice significantly reduces people's willingness to admit ignorance, even when the AI provides incorrect answers and accuracy is incentivized. Participants answered more questions but were less accurate, while their confidence nearly doubled.

Research involving over 3,000 participants across five experiments investigated how AI advice influences human judgment, particularly the willingness to admit uncertainty. The study deliberately provided incorrect AI advice to isolate the effect of AI presence from its accuracy. The findings revealed that merely having access to AI, whether actively requested or passively displayed, drastically diminished participants' inclination to decline answering difficult questions. Consequently, individuals attempted more questions but achieved significantly lower accuracy, while their self-reported confidence paradoxically almost doubled. Even when accuracy was incentivized and inaccuracy penalized, participants still sought and followed AI advice more often and suspended judgment less frequently than when AI was unavailable. This suggests that ubiquitous AI suggestions may not only impact answer correctness but also fundamentally alter the metacognitive threshold for deciding one's own knowledge.

Why it matters

Understanding the cognitive biases introduced by AI assistance is crucial for designing human-AI collaboration systems that promote critical thinking and prevent over-reliance, especially in high-stakes decision-making environments.

How to implement this in your domain

  1. 1Design AI interfaces that explicitly encourage users to critically evaluate AI suggestions.
  2. 2Implement mechanisms for users to easily flag or question AI advice, fostering a culture of skepticism.
  3. 3Train employees on the limitations of AI and the importance of human oversight and judgment.
  4. 4Develop AI systems that can express uncertainty or provide confidence scores, rather than always giving a fluent answer.

Original post by Chiara Marcoccia, Walter Quattrociocchi, Valerio Capraro

"arXiv:2607.13562v1 Announce Type: new Abstract: Knowing when to say "I don't know" is fundamental to human judgment, yet AI assistants offer a fluent answer to almost any question. In five experiments (N = 3,132; four preregistered, one direct replication), participants answered…"

View on X

Originally posted by Chiara Marcoccia, Walter Quattrociocchi, Valerio Capraro on X · view source

Want to go deeper?

Turn these trends into skills with Learnijoy's hands-on AI & tech courses.

Explore courses