LLM Deployment Lacks Operational Evidence in Fraud Detection.

This paper highlights a critical deficiency in the public discourse surrounding the deployment of Large Language Models (LLMs) for sensitive applications like fraud detection, scam investigation, and content moderation. While LLMs are frequently proposed for these trust-and-safety workflows, the existing literature predominantly evaluates them based on model performance metrics, neglecting crucial operational considerations. The researchers conducted a comprehensive survey of 49 operationally relevant sources, supplemented by 15 contextual references, focusing on LLM use in these domains. A key finding is a significant imbalance in the evidence presented: while fraud detection supplies a large portion of the task-specific corpus, content moderation papers offer more explicit public evidence on practical aspects such as latency, cost, governance, and fairness. Specifically, none of the 18 fraud and investigation sources reviewed reported clean per-decision latency, per-decision dollar cost, or calibration evidence. Instead, they primarily focused on offline task performance, retrieval gains, or case-study accuracy. To address this, the paper introduces FORTE, a framework for organizing LLM roles (e.g., classifiers, reviewer assistants), and a minimum deployment-evidence checklist covering essential metrics needed to justify live workflow integration.