Uncertainty-Guided Coreset Selection Meets Model Compression for Efficient Spam Detection

Transformer-based spam detection achieves strong performance but requires large training sets and heavy inference-time models. Recent work (Hassan and Shinde, 2025) showed that uncertainty-and-density-guided coreset selection over SBERT embeddings can match full-data training with as little as 5% of samples, using a low-uncertainty ("Bottom-K") selection strategy. We revisit this idea using only classical TF-IDF features and a lightweight logistic-regression proxy, and find the opposite pattern at moderate coreset sizes: from 25-50% of training data, selecting high-uncertainty ("Top-K") samples consistently outperforms low-uncertainty selection on two benchmark spam datasets (UCI SMS Spam Collection and Ling-Spam), confirmed across 3 random seeds on both datasets, while low-uncertainty selection remains competitive only at very small coreset sizes (≤10%). We further show that models trained on these coresets tolerate aggressive post-hoc compression -- magnitude pruning combined with weight quantization -- with negligible or no F1 loss, though the two datasets diverge sharply in how much compression they tolerate before collapsing. Our results suggest uncertainty-based data efficiency and model compression compose well, but that recommendations from coreset-selection literature are conditional on the strength of the uncertainty proxy used, not universal defaults.