LLM-Generated Autobiographies Show High Confabulation Rates

A unique scene-level audit investigated the factual accuracy of LLM-generated autobiographies by comparing them against a subject's documented life. The study involved generating a 366-day "page-a-day" book using a conversational LLM, with minimal inputs like a template and daily quotes, but not the subject's full corpus. Each day was then audited against an independent verification corpus using a four-level rubric. The audit revealed an alarming 96.7% verification failure rate, meaning 354 out of 366 days contained no positively corroborated scenes. Only 12 days had corroborated content, while 19 days (5.2%) contained claims actively contradicted by the record. The predominant failure mode was identified as "grounded drift," where real individuals, employers, and settings were placed within entirely invented scenes. Independent re-rating confirmed the high failure rate, though it also highlighted fair-to-moderate reliability for the four-way taxonomy used. Regenerating the same days with current named models under identical inputs still resulted in 100% verification failure. While grounding the generation in the subject's actual corpus significantly improved the verification rate, substantial residual confabulation remained, with an 83.3% failure rate even with grounding.