Human-on-the-Loop AI Reduces Legal Discovery Malpractice Risk

This research addresses the critical issue of compounding errors in autonomous Large Language Model (LLM) agents used in electronic discovery (e-discovery), where such errors can lead to legal malpractice. The paper identifies a specific failure mode termed "trajectory collapse," where an early misclassification silently propagates through multi-step reasoning chains, potentially invalidating an entire privilege review. To combat this, the authors make three key contributions. First, they provide a structured taxonomy of agentic failures in legal information retrieval, categorized by functional stage. Second, they introduce a robust four-layer verification architecture designed to intercept these failures before they escalate. This architecture spans planning, reasoning, execution, and uncertainty quantification, ensuring checks at multiple points in the workflow. Third, a preliminary simulation study on a synthetic e-discovery corpus demonstrates the effectiveness of mandatory Human-on-the-Loop (HOTL) escalation thresholds. The results indicate that by implementing calibrated uncertainty thresholds, the risk of privilege waiver can be reduced by up to 61% compared to fully autonomous deployments. Crucially, this significant risk reduction is achieved while routing fewer than one-quarter of the documents to attorney review, highlighting an efficient balance between automation and human oversight.