New Metrics for External Clustering Validation Unify Criteria

When evaluating clustering algorithms against known class labels, scalar metrics are commonly used but often fail to capture a crucial trade-off: a good clustering should be informative about the underlying classes while simultaneously avoiding excessive fragmentation. This research introduces novel normalized scores for cluster homogeneity and parsimony, designed to quantify this inherent trade-off. These new scores are built upon the information bottleneck principle, but with a modification to ensure they do not reward lossy compression. The authors demonstrate, through examples and mathematical proofs, that their definitions of these scores exhibit an intuitive monotonic variation under cluster refinement, a property often lacking in related proposals. Extending the information-theoretic framework beyond traditional Shannon entropies, the study further derives set-matching and pair-based counterparts for both homogeneity and parsimony scores. This extension serves to unify various commonly used evaluation criteria. Notably, in the pair-based setting, the homogeneity-parsimony trade-off is shown to recover the receiver operating characteristic (ROC) curve of binary classifiers. The utility of this framework is illustrated for tasks such as feature selection and algorithm comparison, demonstrating how considering these scores jointly can clarify clustering operating points and help identify Pareto-optimal solutions.