Read-Only Zero-Shot Classifier Expansion from Pairwise Semantics
Abstract
Image-free Zero-Shot Learning (I-ZSL) aims to extend a pre-trained classifier to unseen classes without using training images or image features during adaptation. Existing I-ZSL methods often depend on pre-defined class descriptions and fixed text encoders, which can be misaligned with the classifier space of the deployed model. We propose Geometry-Regularized Relational Weight Synthesis (GeoRWS), a read-only classifier expansion framework that learns to synthesize unseen-class classifier weights from pairwise semantic relations and observed seen-class weights. Given class-pair affinity descriptions generated by a frozen large language model, GeoRWSuses an adaptive semantic encoder to estimate affinity coefficients over seen classifiers through leave-one-out reconstruction. The training objective further includes a classifier-geometry regularizer, which anchors the learned coefficients to neighborhoods in the seen-class classifier head, and a semantic-consistency regularizer, which improves stability under perturbations of pairwise semantic descriptions. At inference, GeoRWS synthesizes unseen-class weights as mixtures of seen-class weights and injects them into the classifier head while keeping the feature extractor fixed. Experiments on standard ZSL and GZSL benchmarks show consistent improvements over image-free and adapted zero-shot baselines, especially on fine-grained recognition tasks.