LLMs Exhibit Authority Bias in Academic Paper Recommendations

As Large Language Models (LLMs) are increasingly employed as conversational search engines for academic literature, a critical question arises: do they evaluate papers based on their content or on external authority signals? This study causally investigates "authority bias," defined as a systematic preference for papers based on factors like author prestige, publication venue, and citation count, rather than the intrinsic quality of their content. Researchers conducted experiments across eight different LLMs (five open-weight and three frontier closed-weight models). They held the title and abstract of academic papers constant while manipulating authority metadata across three conditions: original, flipped (e.g., prestigious author replaced with unknown), and boosted. The results unequivocally showed that authority bias is substantial and directional, meaning LLMs consistently favored papers with higher authority signals. Furthermore, the degree of this bias varied significantly among the models tested. While prompt-level debiasing instructions could partially reduce the bias, they did not eliminate it entirely. The study also uncovered a "say-do gap": debiasing instructions were more effective at suppressing explicit mentions of authority than at altering the underlying behavioral preference for authoritative papers. This suggests that surface-level audits might underestimate the true extent of behavioral bias in these systems.