New Protocol Evaluates Temporal Fidelity of Synthetic Sequential Data

This paper highlights a critical flaw in how synthetic sequential tabular data is often evaluated: conventional methods, which treat records as static distributions, fail to capture temporal inconsistencies. Generative models might produce data that looks statistically similar but contains timestamps running backward, repeated entries, or entity paths that are entirely unrealistic. To address this, the authors propose a new, taxonomy-guided evaluation protocol. The protocol begins by characterizing each dataset based on four properties: time representation, sampling regularity, trajectory dependencies, and schema links. These properties then dictate which specific evaluation dimensions are meaningful. The framework measures several aspects of temporal fidelity, including the validity of timestamps, the cross-sectional structure at aligned time points, the dynamics within individual entities, and how relational structures evolve over time. It also redefines utility and privacy evaluation to focus on trajectories rather than isolated rows. Applying this protocol to eight generative models across thirteen datasets revealed significant discrepancies between rankings based on conventional evaluation and those based on temporal evaluation. The observed failures were systematic, not random, underscoring that temporal fidelity must be directly measured along the time axis.