Latency Is Part of the World Model
Abstract
A world model represents the state of a physical environment and predicts its evolution from sensor measurements. Transduction, propagation and actuation displace the instant at which an event reaches each stream, hence samples sharing an index can describe distinct events. Co-indexed objectives fix the cross-stream offset at zero, replacing a physical property of the sensing apparatus with an inductive bias. We propose LODES, for learned offset distributions on edges between streams, a joint-embedding predictor learning the distributions and a directional influence per edge. One prediction step at evaluation returns a future latent state and the directed timing graph, and correlation over candidate offsets makes the recovered offset follow a test-time displacement on seven sensing systems. Re-aligning a desynchronised stream at the recovered offset restores frozen-probe accuracy, which enables a predictive state to correct its timing without labels.