Timezone: »
There is great interest in generalizing deep learning to more exotic types of data, such as graphs, chemical structures, volumetric images, omndirectional images, etc. In each case, the data has nontrivial structure and symmetries and the challenge is to find the right generalization of classical neural network layers like convolution to reflect this. It has become clear that in all of these cases and more, equivariance to symmetry transformations is the key principle that points us to an effective generalization.
New architectures inspired by this principle have already proved their effectiveness in multiple domains. However, some of the underlying ideas are still foreign to much of the community, partly because of the mathematics involved. The purpose of this tutorial is to bridge this gap by giving a very accessible introduction to this emerging area with many practical examples and details of how to implement equivariant architectures in existing deep learning frameworks.
Timetable: Part I (Taco Cohen) 0:00  Introduction to equivariant networks 39:00  Examples and applications 51:00  Equivariant convolutions
Part II (Risi Kondor) 0:00  Introduction 7:50  Group Representations 27:35  Designing equivariant Neurons 45:30  Fourier theory 56:25  Implementation
Author Information
Risi Kondor (Flatiron Institute)
Risi Kondor joined the Flatiron Institute in 2019 as a Senior Research Scientist with the Center for Computational Mathematics. Previously, Kondor was an Associate Professor in the Department of Computer Science, Statistics, and the Computational and Applied Mathematics Initiative at the University of Chicago. His research interests include computational harmonic analysis and machine learning. Kondor holds a Ph.D. in Computer Science from Columbia University, an MS in Knowledge Discovery and Data Mining from Carnegie Mellon University, and a BA in Mathematics from the University of Cambridge. He also holds a diploma in Computational Fluid Dynamics from the Von Karman Institute for Fluid Dynamics and a diploma in Physics from Eötvös Loránd University in Budapest.
Taco Cohen (Qualcomm AI Research)
Taco Cohen is a machine learning research scientist at Qualcomm AI Research in Amsterdam and a PhD student at the University of Amsterdam, supervised by prof. Max Welling. He was a cofounder of Scyfer, a company focussed on active deep learning, acquired by Qualcomm in 2017. He holds a BSc in theoretical computer science from Utrecht University and a MSc in artificial intelligence from the University of Amsterdam (both cum laude). His research is focussed on understanding and improving deep representation learning, in particular learning of equivariant and disentangled representations, dataefficient deep learning, learning on nonEuclidean domains, and applications of group representation theory and noncommutative harmonic analysis, as well as deep learning based source compression. He has done internships at Google Deepmind (working with Geoff Hinton) and OpenAI. He received the 2014 University of Amsterdam thesis prize, a Google PhD Fellowship, ICLR 2018 best paper award for “Spherical CNNs”, and was named one of 35 innovators under 35 in Europe by MIT in 2018.
Related Events (a corresponding poster, oral, or spotlight)

2020 Tutorial: (Track2) Equivariant Networks Q&A »
Tue. Dec 8th 10:00  10:50 AM Room
More from the Same Authors

2021 : ATOM3D: Tasks on Molecules in Three Dimensions »
Raphael Townshend · Martin Vögele · Patricia Suriana · Alex Derry · Alexander Powers · Yianni Laloudakis · Sidhika Balachandar · Bowen Jing · Brandon Anderson · Stephan Eismann · Risi Kondor · Russ Altman · Ron Dror 
2022 : Multiresolution Mesh Networks For Learning Dynamical Fluid Simulations »
Bach Nguyen · Truong Son Hy · Long TranThanh · Risi Kondor 
2022 : Predicting DrugDrug Interactions using Deep Generative Models on Graphs »
Khang Ngo · Truong Son Hy · Risi Kondor 
2022 : On the Expressive Power of Geometric Graph Neural Networks »
Cristian Bodnar · Chaitanya K. Joshi · Simon Mathis · Taco Cohen · Pietro Liò 
2022 : Deconfounded Imitation Learning »
Risto Vuorio · Pim de Haan · Johann Brehmer · Hanno Ackermann · Daniel Dijkman · Taco Cohen 
2022 : Panel Discussion I: Geometric and topological principles for representation learning in ML »
Irina Higgins · Taco Cohen · Erik Bekkers · Nina Miolane · Rose Yu 
2022 : On the Expressive Power of Geometric Graph Neural Networks »
Cristian Bodnar · Chaitanya K. Joshi · Simon Mathis · Taco Cohen · Pietro Liò 
2022 : From Equivariance to Naturality »
Taco Cohen 
2022 Poster: A PACBayesian Generalization Bound for Equivariant Networks »
Arash Behboodi · Gabriele Cesa · Taco Cohen 
2022 Poster: Weakly supervised causal representation learning »
Johann Brehmer · Pim de Haan · Phillip Lippe · Taco Cohen 
2022 Poster: On the symmetries of the synchronization problem in CryoEM: MultiFrequency Vector Diffusion Maps on the Projective Plane »
Gabriele Cesa · Arash Behboodi · Taco Cohen · Max Welling 
2021 Poster: Autobahn: Automorphismbased Graph Neural Nets »
Erik Thiede · Wenda Zhou · Risi Kondor 
2021 : ATOM3D: Tasks on Molecules in Three Dimensions »
Raphael Townshend · Martin Vögele · Patricia Suriana · Alex Derry · Alexander Powers · Yianni Laloudakis · Sidhika Balachandar · Bowen Jing · Brandon Anderson · Stephan Eismann · Risi Kondor · Russ Altman · Ron Dror 
2020 Poster: Natural Graph Networks »
Pim de Haan · Taco Cohen · Max Welling 
2019 : Solutions »
Fitzroy Christian · Lily Hu · Risi Kondor · Brandeis Marshall · Fabian Rogers · Friederike Schuur · Emanuel Moss 
2019 Workshop: Minding the Gap: Between Fairness and Ethics »
Igor Rubinov · Risi Kondor · Jack Poulson · Manfred K. Warmuth · Emanuel Moss · Alexa Hagerty 
2019 Poster: Cormorant: Covariant Molecular Neural Networks »
Brandon Anderson · Truong Son Hy · Risi Kondor 
2019 Spotlight: Cormorant: Covariant Molecular Neural Networks »
Brandon Anderson · Truong Son Hy · Risi Kondor 
2019 Poster: A General Theory of Equivariant CNNs on Homogeneous Spaces »
Taco Cohen · Mario Geiger · Maurice Weiler 
2018 Poster: Clebsch–Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Network »
Risi Kondor · Zhen Lin · Shubhendu Trivedi 
2018 Poster: 3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data »
Maurice Weiler · Wouter Boomsma · Mario Geiger · Max Welling · Taco Cohen