LLMs Prefer Critically Acclaimed Films Over Commercial Success

This research investigates how large language models (LLMs) evaluate cultural artifacts, specifically films, given their training on vast corpora containing human judgments. The study aimed to determine if LLMs systematically reproduce evaluative hierarchies, either mirroring popular internet trends or reflecting critical prestige. Researchers tested eight models from four families (Anthropic, OpenAI, Alibaba, Mistral) using a benchmark of 200 films categorized by critical acclaim, commercial success, or both. Across 20,000 pairwise comparisons per model, a consistent "critical acclaim orientation" was observed: LLMs preferred critically acclaimed yet commercially obscure films over commercially successful but critically unrecognized ones. This preference intensified with increasing model scale within each family. The study also found that public visibility and popular reception distinctly explain preferences, and that different prompt framings (evaluative vs. recommendation-oriented) produced divergent rankings, suggesting that this critical bias can manifest indirectly in real-world applications.