Spokes, not a Ring: Testing Continuity of Representation Manifolds

Recent work has hypothesised that large language models may represent some concepts as multi-dimensional manifolds instead of single directions. Manifold-aware probing and steering methods rely on this hypothesis and assume that a path along the manifold corresponds to continuous changes in the represented concept and resulting behaviour. In this work, we test this continuous manifold assumption on the prototypical days-of-the-week manifold. Surprisingly, we find that the weekday representations are better described as seven discrete states arranged in a circle than as a continuous ring. First, we show that steering sensitivity is non-uniform on and around the ring, resulting in a step-like mapping between representation and behaviour rather than a smooth one. Second, we find that activations cluster at seven discrete points along the ring and that neither natural text nor time of day systematically occupy the gaps. We propose our sensitivity and occupancy tests as explicit tests of continuity that should be applied to other proposed representation manifolds before they are relied on for steering.