New Method Certifies Multi-Agent Reliability Without Independence Assumption

Traditional methods for assessing the compositional reliability of multi-agent systems often rely on the assumption that component failures are conditionally independent. This paper rigorously tests and refutes this assumption, revealing a significant flaw in how redundancy is typically credited. In a preregistered evaluation of 18,000 missions, two instances of the same model in a handoff scenario co-failed on 90.0% of missions where either failed, indicating a strong positive dependence (phi 0.916). This means that when components share a model, joint failure rates are inflated beyond what independence would predict, leading to an overestimation of redundancy benefits. The research highlights that simply substituting a different model can reduce this association, while substituting a different vendor with an already different model did not show further reduction. The authors also demonstrate that fitting a dependence model is problematic, as a bootstrap bound on its functional loses coverage with more data, making such certificates unreliable. To address this, the paper proposes a novel finite-sample certificate that assumes no dependence structure. This certificate uses a linear program over the joint distribution, bounded by a Bonferroni-Clopper-Pearson box around measured co-execution moments. This method is proven to be sound, sharp for the information supplied, and monotone in the moment family. Enriching the moment functionals from ten to fourteen significantly narrowed the identified interval by 85.7%, lifting the certified floor of reliability from 0.2455 to 0.4116. An anytime-valid companion certificate also maintains a controlled Type-I error rate.