Poster
in
Workshop: Symmetry and Geometry in Neural Representations (NeurReps)

Training shapes the curvature of shallow neural network representations

Jacob Zavatone-Veth · Julian Rubinfien · Cengiz Pehlevan

Keywords: Neural Networks Riemannian geometry Curvature

Project Page [ Poster] [ OpenReview]

Abstract

We study how training shapes the Riemannian geometry induced by neural network feature maps. At infinite width, shallow neural networks induce highly symmetric metrics on input space. Feature learning in networks trained to perform simple classification tasks magnifies local areas and reduces curvature along decision boundaries. These changes are consistent with previously proposed geometric approaches for hand-tuning of kernel methods to improve generalization.

Chat is not available.