The Interface Contract: Repairable Geometry and Partially Predictable Drift in Continual Representation Learning
Atharva Airen ⋅ Anirban Pal
Abstract
Continual learning updates shift feature representations, causing a fixed classifier to fail even when a newly retrained classifier can recover lost accuracy. Using branches cloned from byte-identical training states, we attribute the additional drop to training on the incoming task rather than continuing the previous task. The change is largely reversible but not rigid: ridge-affine maps recover 84--90\% of the drop (mean clipped recovery over seed--boundary units for MLP and linear classifiers), outperforming orthogonal maps. What can be predicted depends on what is measured: later features are not determined by a few early updates, and extrapolating those updates is harmful. A classifier-focused method, selected on development seeds and applied to 20 held-out class orders without endpoint fitting, endpoint labels, or retuning, lowers cross-entropy (paired $\Delta \approx -0.94$, 95\% CI $[-1.13, -0.78]$) while leaving top-1 accuracy statistically unchanged. Thus, we predict cross-entropy for one linear classifier, not all endpoint features; the target, available early state, and metric must be specified. This separates reduced information available to a classifier from classifier--feature misalignment.
Chat is not available.
Successful Page Load