Tokenized Neural Fields: Structured Representations of Continuous Signals

We introduce Tokenized Neural Fields (TNF), a unified framework for representing continuous signals through a compact set of learnable tokens. Unlike encoder-based pipelines or global latent codes, TNF provides a structured tokenization in which individual tokens specialize to distinct aspects of a signal and interact with coordinate queries via cross-attention. This decoupling of representation from decoder architecture enables scalable training across modalities, efficient adaptation to new signals, and a natural basis for probabilistic inference in token space. We validate TNF across 1D function regression, 2D image reconstruction, and 3D scene modeling, showing that tokenized representations achieve superior fidelity with fewer parameters compared to encoder- or latent-based baselines. Beyond accurate reconstructions, TNF tokens exhibit emergent specialization, support interpolation and morphing, and enable generative modeling when paired with diffusion transformers. Together, these results highlight tokenization as a powerful paradigm for bridging implicit neural representations with the structured inference and generative capabilities increasingly central to large foundation models.