New Verifier Ensures Source-Aware Factuality for LLM Agents

This research introduces ProvenanceGuard, a new verification system specifically designed for LLM agents that utilize the Model Context Protocol (MCP) to synthesize information from multiple heterogeneous sources. The core problem it addresses is "cross-source conflation," where an AI agent might generate a factual claim that is supported by *some* evidence, but incorrectly attributes it to the wrong source. ProvenanceGuard operates by analyzing MCP traces, breaking down agent answers into atomic claims, and then routing each claim to its correct source-specific evidence. It employs Natural Language Inference (NLI) and token-alignment to verify support and compares the agent's stated attribution with the actual supporting source. The system provides per-claim verdicts and an overall allow/block decision for the answer, with blocked answers capable of being repaired and re-verified. Evaluated on medical-domain MCP-agent traces, ProvenanceGuard achieved strong performance in detecting and preventing attribution errors. It significantly outperformed source-blind baselines, demonstrating its effectiveness in ensuring accurate source attribution, especially in complex multi-source environments.