Geometry-Informed Tracing of Model Adaptation Path under Protected-Capability Constraints

Model adaptation specializes a pretrained model under a constraint on degradation of a protected pretrained capability. Standard workflows train and evaluate a sparse set of candidate models with varying specialization v.s. pretrained capability trade-off, which could miss feasible models near the binding protected capacity degradation constraint. We instead study the model adaptation path: the entire set of adapted models under varying trade-offs. We show the model adaptation path is affine under a special case; more generally, we give a closeness law that bounds the curvature of the adaptation path. This geometry motivates two model adaptation tracing algorithms: a one-run ray from a single endpoint and a proximal chain of locally anchored solves for smoothly curved paths. We numerically test these tracing algorithms across model adaptations in instruction following, biomedical question answering, and code generation. Our algorithms recover feasible candidates missed by independent sweeps, Bayesian and multiobjective search, and early stopping in tight-budget regimes, while enjoying less computational cost.