SUPERVISE: A Unified Framework for Standardized and Reproducible Superpixel Evaluation

Image segmentation into superpixels is a widely used technique in computer vision, with a large body of work developed over the years. Yet, its evaluation remains poorly standardized, as methods are compared under heterogeneous protocols, across inconsistent scale ranges, and with redundant or misaligned metric subsets. In this work, we introduce $\textbf{SUPERVISE}$ (SUperpixel PERformance VISualization and Evaluation), a unified and reproducible benchmark that addresses these longstanding inconsistencies. Built on a scale-normalized protocol based on interpolation over the actual number of generated superpixels, SUPERVISE enables fair and consistent cross-method comparisons. We leverage this standardized framework to conduct the first large-scale statistical analysis of 20 evaluation metrics across 7 datasets and 31 methods — revealing strong inter-metric correlations and significant redundancy that challenge common evaluation practices. These findings provide empirical grounding for a compact, representative metric subset. SUPERVISE is released as a lightweight, modular framework built on precomputed shared label maps, making large-scale reproducible evaluation accessible to the community. All code, label maps, and evaluation results are publicly available at: https://anonymous.4open.science/r/evaluation_superpixel-08B1/