High-dimensional data representation is in a confused infancy compared to statistical decision theory. How to optimize kernels or so called feature vectors? Should they increase or reduce dimensionality? Suprisingly, deep neural networks have managed to build kernels acculumating experimental successes. This lecture shows that invariance emerges as a central concept to understand high-dimensional representations, and deep network mysteries.
Intra-class variability is the curse of most high-dimensional signal classifications. Fighting it means finding informative invariants. Standard mathematical invariants are either non-stable for signal classification or not sufficiently discriminative. We explain how convolution networks compute stable informative invariants over any group such as translations, rotations or frequency transpositions, by scattering data in high dimensional spaces, with wavelet filters. Beyond groups, invariants over manifolds can also be learned with unsupervised strategies that involve sparsity constraints. Applications will be discussed and shown on images and sounds.
Stephane Mallat (Ecole Polytechnique Paris)
Stéphane Mallat received the Ph.D. degree in electrical engineering from the University of Pennsylvania, in 1988. He was then Professor at the Courant Institute of Mathematical Sciences. In 1995, he became Professor in Applied Mathematics at Ecole Polytechnique, Paris. From 2001 to 2007 he was co-founder and CEO of a semiconductor start-up company. In 2012 he joined the Computer Science Department of Ecole Normale Supérieure, in Paris. Stéphane Mallat’s research interests include signal processing, computer vision, harmonic analysis and learning. He wrote a “Wavelet tour of signal processing: the sparse way”. In 1997, he received the Outstanding Achievement Award from the SPIE Society and was a plenary lecturer at the International Congress of Mathematicians in 1998. He also received the 2004 European IST Grand prize, the 2004 INIST-CNRS prize for most cited French researcher in engineering and computer science, and the 2007 EADS grand prize of the French Academy of Sciences.
More from the Same Authors
2014 Poster: Unsupervised Deep Haar Scattering on Graphs »
Xu Chen · Xiuyuan Cheng · Stephane Mallat
2013 Workshop: Neural Information Processing Scaled for Bioacoustics : NIPS4B »
Hervé GLOTIN · Yann LeCun · Thierry Artières · Stephane Mallat · Ofer Tchernichovski · Xanadu Halkias