Shaping Latent Geometry with Noise-Injected Hopfield Dynamics

Latent representations that exhibit geometric structure are central to robust and generalizable learning. While such geometry often emerges incidentally, previous work has attempted to enforce it through regularization or architectural changes. In this work, we propose the Noise-Injected Hopfield Retrieval (NIHR) layer, a differentiable module that injects Gaussian noise into the update dynamics of Modern Hopfield Networks to actively shape latent space geometry. By controlling the number of retrieval iterations and the inverse temperature, NIHR enables a tunable transition between discrete attractors and smooth continuous manifolds. When integrated into autoencoding and classification pipelines, NIHR consistently improves robustness to corruptions, latent structure quality, and linear separability. Our results suggest that NIHR provides an effective mechanism for imposing meaningful geometric inductive biases in neural representations without auxiliary loss functions.