In-distribution skill does not diagnose transfer: a controlled ocean testbed for scientific representations

Scientific representations are commonly selected by in-distribution predictive accuracy, but this need not determine transfer after physical regime change. We compare four data-driven predictors of the same five-channel North Atlantic ocean state: Euclidean and spatially regularized regression, low-rank EOF/POP dynamics, and a nonlinear latent dynamical emulator. All are trained on preindustrial control data and evaluated at 1-, 5-, and 10-year leads under historical and SSP3-7.0 forcing. Their in-distribution and out-of-distribution rankings are reversed: the nonlinear emulator is modestly worse on preindustrial test data at long leads but generalizes best after a late-century reorganization marked by near-suppression of Labrador Sea deep convection. POP is intermediate, while direct regressions fail most strongly. Attribution analysis suggests that state-evolution objectives preserve multivariate physical relationships weakly constrained by the scalar prediction task but relevant after regime change. These results motivate evaluating scientific representations across physically meaningful dynamical shifts, not only held-out samples from the training regime. These results suggest when both regimes cannot be sampled, predictors should preserve generalizable physical mechanisms and extrapolate along a physically plausible manifold.