New Metric Diagnoses Dependency Gaps in Synthetic Tabular Data

Synthetic tabular data is highly valued for its ability to preserve not only individual column distributions but also the complex dependencies between columns. These inter-column dependencies are critical for maintaining the discriminative signal, especially for minority classes in imbalanced datasets like those found in fraud detection or clinical risk assessment. However, commonly used metrics for evaluating synthetic data often fail to adequately capture this crucial aspect. This study introduces a novel, dependency-aware fidelity diagnostic designed to expose a "dependency gap" in synthetic tabular data. The diagnostic decomposes a strong classifier two-sample test (XGB-C2ST) into marginal, dependency, and numerical-categorical cross components. By anchoring this test between a worst-case scenario (all dependency destroyed) and a best-case (real data), it reveals a significant dependency gap that standard metrics overlook. Applying this to a state-of-the-art flow-matching generator, the research found that destroying dependency collapses minority-class utility, and the generator's residual gap consistently carries a utility cost. Surprisingly, this gap persists even with a 16x increase in model capacity, suggesting it stems from a lack of direct dependency supervision rather than capacity limits. This highlights that simple interventions or increased capacity alone do not close this higher-order dependency gap, cautioning against over-reliance on such fixes.