When Scores Do Not Align: Diagnosing Calibration Failure in Large-Scale Open-Set Recognition

Generalized zero-shot and open-set recognition must rank classes whose training evidence ranges from abundant to completely absent. A common approach reconciles supervised seen-class scores and zero-shot unseen-class scores using a single additive calibration constant. We show that this assumption breaks down in a 17,393-class fine-grained recognition task containing 11,598 classes with no training images. On an offline 900-class validation harness, the zero-shot branch achieves 0.1198 accuracy in isolation, but naive merging yields only 0.0113 unseen accuracy and 0.4402 overall accuracy. We diagnose the failure as a mismatch in score shape rather than location. Supervised scores are concentrated, whereas zero-shot scores are diffuse, and a scalar shift cannot alter variance, concentration, or tail behavior. Homogeneous shared-space scoring, using identical underlying heads, improves overall accuracy to 0.4737 and unseen accuracy to 0.1265. A nine-signal logistic router further improves routing AUROC from 0.8506 to 0.8749 and yields 163 additional correct predictions on the real 35,665-image evaluation, although an alternative offline-selected configuration regresses below the no-routing baseline after deployment. These results show that calibration should diagnose score geometry before applying scalar corrections and that pseudo-unseen validation should be treated as a proxy rather than a guarantee of deployment performance.