Tunnel Room: Probing What JEPA World Models Preserve

Joint-Embedding Predictive Architectures (JEPAs) learn representations that abstract away visual details unnecessary for prediction, but which information is preserved or discarded remains poorly understood. We introduce Tunnel Room, a controlled navigation benchmark that probes this question by making a small visual cue, such as color or shape, determine where a tunnel leads. We study LeWorldModel (LeWM) and find a consistent separation between learning where tunnels are and where they lead. On Tunnel Room Color, linear probes recover tunnel locations with mean $R^2=.934$, but achieve only $.224$ when the same coordinates are identified by color. Reconstructions and goal-energy landscapes similarly preserve tunnel geometry. The effect extends to prediction and planning, where planners favor the nearest tunnel even when it is incorrect, and cue swaps that lead to different destinations produce substantially less separation in predicted than observed features. These results highlight a challenge for JEPA-based world models: information that is visually small may nevertheless be critical for predicting the consequences of actions. Why models develop this particular representational asymmetry remains an open question.