Bifurcations from Weight Space: JEPAWG on Parametric Dynamical Systems

Trained network weights are increasingly treated as data, but there is rarely ground truth about what a population of models should encode, which hampers systematic validation of weight-space methods. This makes parametric dynamical systems a natural testbed: the dynamics change qualitatively at submanifolds of parameter space that are, in well-studied families, known analytically or numerically. Fine-tuning a forecasting network at each point of a parameter grid yields a model zoo indexed by a physical parameter, against which weight-space diagnostics can be checked. We build such zoos for four dynamical systems using JEPAWG, a recently proposed joint-embedding hypernetwork built around a learned parameter–weight representation. It was introduced for lattice field theories, where its latent geometry located the Ising phase transition and estimated its critical exponent without simulation. Applied to dynamical systems, JEPAWG is competitive on forecasting with a hypernetwork trained directly for that task, despite being supervised on the parameter–weight correspondence rather than on forecasts. Beyond forecasting, the norm of the parameter-derivative of the decoded weights locates Hopf points and curves, tracks a period-doubling cascade closely enough to recover Feigenbaum scaling, and detects a global bifurcation, which has no eigenvalue signature and is therefore invisible to linearization. This works because the decoder cannot represent per-run training noise: it returns a regularized counterpart of the zoo, whose intrinsic dimension tracks the control manifold's. Together, these results indicate that a learned parameter--weight latent is a workable instrument for reading physical structure off a population of trained networks. We also delimit what the construction cannot see.