Two Invariances Make a 2048-Dimensional PAC-Bayes Certificate Scalar
Keisuke Yokota
Abstract
Soft-prompt tuning adapts a frozen vision-language model by moving 2048 parameters from a few labelled examples per class. What blocks a PAC-Bayes certificate is the accounting, not the fit: prior, posterior, search and numerical risk evaluation must all be stated in that space and paid for. We learn the full update on one split of the task's own examples and let a second, independent split decide only how far a stochastic predictor travels along it. Two invariances then make the certificate exact rather than approximate. Relative entropy is unchanged by the injective affine map from the coordinate to a prompt, so the complexity charged in 2048 dimensions *is* a scalar complexity. And the empirical risk is blind to how a posterior arranges mass *within* a level set of the calibration error count, while the relative entropy is least when that mass matches the prior, so the optimisation descends losslessly to a simplex of dimension at most $n$. The line is anchored exactly, one end reproducing zero-shot CLIP and the other the fitted prompt. From sixteen labelled examples per class, every certified class-balanced Gibbs risk on eight CLIP tasks falls below random-guess error. The geometry is that of a parameter path, not of a learned representation; we claim no equivariance.
Chat is not available.
Successful Page Load