The article "Revisiting Silhouette Aggregation", presented by Assistant Professor John Pavlopoulos at Discovery Science 2024 ιn Pisa, was recognized as Best Paper

Assistant Professor John Pavlopoulos’ latest research, "Revisiting Silhouette Aggregation", was recognized as Best Paper Runner Up at Discovery Science 2024 in Pisa. Dr. Pavlopoulos, in collaboration with A. Likas and G. Vardakas from the Department of Computer Science & Engineering of the University of Ioannina, uncovered a powerful yet overlooked method of evaluating clustering solutions using the Silhouette Coefficient—particularly advantageous for datasets with cluster imbalance. Their paper further introduced a novel sampling technique, essential for large datasets, enhancing robustness and reliability in clustering analysis.

In clustering analysis, Silhouette Coefficient is a commonly used internal evaluation measure, assigning each data point a score that reflects the quality of its clustering assignment. Traditionally, an aggregated Silhouette score for a dataset is calculated through micro-averaging, where the scores of all points are averaged to a single value. However, this study examines the alternative macro-averaging approach, where scores are first averaged within each cluster, and these cluster-level scores are then averaged across clusters.

Through synthetic examples, the study demonstrate that micro-averaging is highly sensitive to cluster imbalance whereas macro-averaging is considerably more robust. Its analysis further reveals that existing uniform sub-sampling techniques negatively impact the robustness of micro- and macro-averaged Silhouette scores in the presence of cluster imbalance.

To address this issue, the proposes a novel per-cluster sampling strategy that maintains the robustness of the macro-averaged Silhouette score. This method is evaluated on eight real-world datasets, where both micro- and macro-averaged Silhouette scores are compared in clustering tasks, revealing the potential benefits of considering both aggregation strategies for more reliable clustering evaluation.

Overall, this work highlights a refined toolset for robust clustering assessment, particularly valuable for large or imbalanced datasets.

Preprint: https://arxiv.org/abs/2401.05831