New Pipeline Audits LLM Data, Uncovers Hidden Alignment Flaws.

The quality of training data is a major bottleneck for aligning Large Language Models (LLMs), as large datasets often contain hidden contradictions, safety issues, and human annotation errors. Traditional auditing methods like deduplication or LLM-as-a-judge often fail to capture the true impact of individual data records or deep functional rule conflicts. Researchers have developed a scalable, inference-only data valuation pipeline that estimates Shapley values without requiring iterative model retraining. This framework maps semantic neighborhoods into a directed graph, evaluating data utility by analyzing a reference LLM's probability distribution using zero-shot and one-shot conditional log-likelihood shifts. The pipeline translates these predictive influence scores into localized advantage metrics to pinpoint gradient-conflicting records. Applied to the HelpSteer2 dataset, it reduced manual audit search space by 99.1% and found numerous falsely-labeled records. On Anthropic's HH-RLHF dataset, it uncovered thousands of hidden safety and factual preference inversions, critically exposing flaws in evaluation benchmarks where capable models are penalized by incorrect human ground-truth labels.