New Clustering Method Improves Accuracy with Density and Graph Data

Clustering is a foundational technique in data mining, used for pattern recognition through unsupervised learning. While various methods exist, including hierarchical, density-based, and graph clustering, traditional approaches often calculate inter-cluster similarity based solely on pairwise distances. This can overlook crucial aspects like density variations and structural connectivity within graphs. To address these limitations, a new method called Density-aware Hierarchical Clustering based on Element-Categorized Connection Subgraphs (DHC-ECS) has been developed. DHC-ECS innovatively combines elements from hierarchical, density-based, and graph clustering paradigms. Its core contribution is a novel inter-cluster similarity metric. This metric goes beyond simple distances, incorporating insights from element categorization within K-Nearest Neighbor (KNN) connection subgraphs, kernel density estimation, and the local connectivity patterns within sub-clusters. Extensive testing on a variety of benchmark datasets demonstrates that DHC-ECS achieves superior overall performance in terms of clustering accuracy and robustness to parameter changes compared to existing state-of-the-art methods. The work highlights the potential of this algorithm for low-dimensional datasets by effectively leveraging both local density and graph-structured connectivity, and it also offers a path towards determining an intrinsic threshold, thereby reducing the need for manual parameter tuning.