Predicting the Performance of Foundation Models via Agreement-on-the-Line

Estimating out-of-distribution performance is critical to safely deploying machine learning models. Baek et al. showed that the phenomenon "agreement-on-the-line" (AGL) can be a reliable method for predicting OOD accuracy of models in an ensemble of CNNs trained from scratch. The current practice is to lightly fine-tune foundation models, but it is unclear whether such fine-tuning can yield the sufficiently diverse models needed for AGL based methods to work. In this paper, we develop methods for reliably applying AGL based OOD estimation to fine-tuned foundation models. In particular, we first study the case of fine-tuning a single foundation model, where we extensively show how different types of randomness contribute to the AGL of the resulting model sets; we find, somewhat surprisingly, that it is typically possible to obtain strong agreement via random initialization of the linear head alone. Next, we study how multiple foundation models, pretrained on different data sets but fine-tuned on the same task may produce agreement; we show, again rather surprisingly, that the diversity of such models is already sufficient and not too disparate for them to all lie on the same agreement line. In total, these methods enable reliable and efficient estimation of OOD accuracy for fine-tuned foundation models, without leveraging any labeled OOD data.