Neural Fields Encode Class-Related Adaptation Geometry

Neural fields are often used to encode observations as points in weight space. We find that neural fields encode information not only in their activations, outputs, or parameter values, but also in the ease with which they can realize neighboring functions. We refer to this property as their adaptation geometry. We study this by initializing neural fields with priors for each class, and classifying observations by selecting the prior which most easily reconstructed them. On linearizing the neural fields at the prior, the adaptation cost is exactly a Mahalanobis distance whose covariance is induced by the prior's empirical tangent kernel. We verify the cost estimated by this linear model closely tracks the relative ordering and decisions of the cost of the full, nonlinear adaptation, and the tangent geometry is not interchangeable across classes. Our results support treating local adaptation geometry as part of a neural representation, distinct from both its parameter point and the downstream utility of a particular adaptation rule.