VERITAS Tool Automates Scientific Research Replication and Verification.

The rapid pace of scientific publication, particularly in AI, has outstripped the capacity for thorough review and independent verification. Manual replication of research is slow and costly, leading to a growing need for automated solutions. While existing coding agents automate parts of this process, they are often confined to specific benchmarks. This paper presents VERITAS, a novel, domain-agnostic replication framework designed to address this gap. VERITAS operates using command-line interface (CLI) coding agents. Given a scientific paper, a code repository, or both, the tool systematically extracts the paper's claims, executes the described methodology, and automatically resolves issues encountered during the process. Finally, VERITAS evaluates each claim against the evidence generated from the experiment runs. The output includes an importance-weighted Replication Score, a detailed log of all applied fixes with severity ratings, and the patched codebase. Evaluated on CORE-Bench and ReplicationBench, covering 65 papers across various scientific fields, VERITAS achieved state-of-the-art performance, outperforming strong baselines on all metrics.