Measured but Not Detected: How Class Assignment Breaks the Zero-Day Detection Rate

The Zero-Day Detection Rate (Z-DR) is widely used to evaluate how an intrusion detector handles attack classes it was not trained on. It measures detection under one condition. When a system labels a flow by thresholding a novelty score, Z-DR is a point on the ROC curve of that score and the two agree by construction: recomputed from the tables published with the metric itself, Z-DR and AUC correlate at r = +0.99. Zero-shot and generalized zero-shot detectors assign a class instead, and have carried the metric with them. There it inverts. Across seventeen held-out attack classes in two loT datasets, Z-DR and AUROC are negatively correlated (r = -0.79 and -0.57): the reported metric improves as the novelty score gets worse. In the clearest case a detector scores 99.98% Z-DR on a score whose AUROC is 0.121, one that ranks unseen attacks as more typical of training traffic than training traffic itself. Two systems with bit-identical novelty scores, and therefore identical AUROC, differ by 26.3 Z-DR points on how known classes are labelled a choice made after the novelty decision and incapable of affecting it. Under a comparison controlled for these effects, the attribution distance in the system we evaluate does not beat the base classifier's own softmax confidence on any of nine held-out classes, a comparison Z-DR had concealed.