RAG Boosts LLM Accuracy for Public Health Questions

A new study explores how Retrieval-Augmented Generation (RAG) can make large language models (LLMs) more reliable for public health question answering, a domain where accuracy and up-to-date information are critical. LLMs often struggle with factual consistency and keeping pace with rapidly changing official guidance, leading to hallucinations. By grounding LLM responses in an explicitly maintained corpus, RAG effectively mitigates these risks. The research extended the PubHealthBench dataset to a retrieval-augmented setting, systematically evaluating various retrieval and generation configurations. Findings indicate that hybrid retrieval, combining dense and sparse methods, consistently enhances recall and ranking quality. Crucially, providing retrieved context substantially boosts multiple-choice accuracy across diverse LLMs, allowing even smaller, open-weight models to perform as well as or better than larger models used without RAG. The study also introduced a rubric-based LLM-as-a-judge evaluation method, validated against human annotations, to assess free-form answers for faithfulness, completeness, clarity, and factual consistency, offering practical guidance for building and evaluating RAG systems in sensitive domains.