Comparing Scoring Signals for Gaussian Coreset Selection Across Visual Benchmarks

Training deep networks on large-scale data is often out of reach for researchers with limited GPU access, making coreset selection — picking a small, representative training subset — a practical necessity rather than a luxury. We compare four scoring signals inside a single Gaussian bin-selection framework: random-weight entropy, an ImageNet-pretrained adaptive scorer, EL2N (early-training prediction error), and DINOv2 feature distance. Across ResNet-18 experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet spanning budgets of 1% to 100% EL2N and DINOv2 consistently beat random selection in the practically relevant 5–30% regime, while ImageNet-borrowed signals lose their edge on harder, more distribution-shifted benchmarks. The result is a simple, evidence-backed recipe for cutting training cost without giving up accuracy.