Timezone: »

 
Understanding Optimization Challenges when Encoding to Geometric Structures
Babak Esmaeili · Robin Walters · Heiko Zimmermann · Jan-Willem van de Meent

@
in Symmetry and Geometry in Neural Representations (NeurReps) »
Event URL: https://openreview.net/forum?id=s3vd1i471Rc »

Geometric inductive biases such as spatial curvature, factorizability, or equivariance have been shown to enable learning of latent spaces which better reflect the structure of data and perform better on downstream tasks. Training such models, however, can be a challenging task due to the topological constraints imposed by encoding to such structures. In this paper, we theoretically and empirically characterize obstructions to training autoencoders with geometric latent spaces. These include issues such as singularity (e.g. self-intersection), incorrect degree or winding number, and non-isometric homeomorphic embedding. We propose a method, isometric autoencoder, to improve the stability of training and convergence to an isometric mapping in geometric latent spaces. We perform an empirical evaluation of this method over 2 domains, which demonstrates that our approach can better circumvent the identified optimization problems.

Keywords: Representation Learning · Autoencoders · Equivariant · Lie groups · topological · homeomorphism · isometry

Author Information

Babak Esmaeili (University of Amsterdam)
Robin Walters (Northeastern University)
Heiko Zimmermann (University of Amsterdam)
Jan-Willem van de Meent (University of Amsterdam and Northeastern University)

More from the Same Authors