Learning Sparse Functional Representations with Neural Operators

Physical fields—vorticity, pressure, temperature—are continuous functions over spatial domains, often observed at varying resolutions and on irregular grids. Sparse Autoencoders (SAEs) have shown promise for learning structured representations, but their fixed-dimensional Euclidean parameterization ties concepts to specific discretizations and encodes only concept presence, making them poorly suited for this setting. We introduce sparse autoencoder neural operators (SAE-NOs), operating in function spaces instead. We formalize the functional representation hypothesis, modeling data as sparse compositions of structured functions, and instantiate it as Fourier neural operators with joint concept and domain sparsity. Concept sparsity selects which concepts are active; domain sparsity governs where and how they are expressed. SAE-FNO learns localized concepts with stable structure across sparsity levels, reuses them across independently processed translated frames, and represents data with fewer effective concepts. Crucially, SAE-FNO generalizes to unseen resolutions where standard SAEs fail.