Character Subspaces Leave No Room for Grokking

Whether a network groks a modular arithmetic task, and how long it waits, moves with capacity. A memorisation timescale and a generalisation timescale are both functions of parameter count, and the window between them closes as the model grows. At the far end of that axis it vanishes together with memorisation itself. For two-layer networks with holomorphic activation $\sigma(z)=z^k$ on roots-of-unity inputs the expressible class is a fixed $(k+1)$-dimensional space of characters of $(\mathbb{Z}_p)^2$, and a target outside it cannot be fitted even on the training set. The obstruction survives the choice of basis. Under an arbitrary learned embedding shared across hidden units the output still has matrix rank at most $k+1$, which leaves $ab$ unreachable at every degree below $p-1$. A classification of this shape was conjectured from experiment by Doshi et al. (2024) in a universal-approximator setting, where failure means memorisation without generalisation. Here it means the training set cannot be fitted. The criterion is exact in both directions, a task being expressible precisely when its Fourier support lies in the $k+1$ frequencies $(\ell,k-\ell)$ on the diagonal $u+v\equiv k \pmod p$, together with a training-loss lower bound independent of hidden width. Across 585 runs the criterion agrees with the outcome 584 times, and no run memorises without generalising. A linear bottleneck in a ReLU network recovers the same three regimes as it narrows, placing the algebraic extreme on the capacity axis. The argument uses only that the group is finite abelian, which makes the nonabelian case, where irreducible representations are no longer one-dimensional, the natural next question.