Timezone: »
Inferring the position of objects and their rigid transformations is still an open problem in visual scene understanding. Here we propose a neuromorphic framework that poses scene understanding as a factorization problem and uses a resonator network to extract object identities and their transformations. The framework uses vector binding operations to produce generative image models in which binding acts as the equivariant operation for geometric transformations. A scene can therefore be described as a sum of vector products, which in turn can be efficiently factorized by a resonator network to infer objects and their poses. We also describe a hierarchical resonator network that enables the definition of a partitioned architecture in which vector binding is equivariant for horizontal and vertical translation within one partition, and for rotation and scaling within the other partition. We demonstrate our approach using synthetic scenes composed of simple 2D shapes undergoing rigid geometric transformations and color changes.
Author Information
Alpha Renner (Institute of Neuroinformatics, University of Zurich and ETH Zurich)
Giacomo Indiveri (University of Zurich and ETH Zurich)

Giacomo Indiveri is a dual Professor at the Faculty of Science of the University of Zurich and at Department of Information Technology and Electrical Engineering of ETH Zurich, Switzerland. He is the director of the Institute of Neuroinformatics of the University of Zurich and ETH Zurich. He obtained an M.Sc. degree in electrical engineering in 1992 and a Ph.D. degree in computer science from the University of Genoa, Italy in 2004. Engineer by training, Indiveri has also expertise in neuroscience, computer science, and machine learning. He has been combining these disciplines by studying natural and artificial intelligence in neural processing systems and in neuromorphic cognitive agents. His latest research interests lie in the study of spike-based learning mechanisms and recurrent networks of biologically plausible neurons, and in their integration in real-time closed-loop sensory-motor systems designed using analog/digital circuits and emerging memory technologies. His group uses these neuromorphic circuits to validate brain inspired computational paradigms in real-world scenarios, and to develop a new generation of fault-tolerant event-based neuromorphic computing technologies. Indiveri is senior member of the IEEE society, and a recipient of the 2021 IEEE Biomedical Circuits and Systems Best Paper Award. He is also an ERC fellow, recipient of three European Research Council grants.
Lazar Supic (University of California Berkeley)
Andreea Danielescu (Accenture Labs)
Bruno Olshausen (Redwood Center/UC Berkeley)
Fritz Sommer (UC Berkeley)
Yulia Sandamirskaya (Intel)
Edward Frady (Intel)
More from the Same Authors
-
2022 : Disentangling Images with Lie Group Transformations and Sparse Coding »
Ho Yin Chau · Frank Qiu · Yubei Chen · Bruno Olshausen -
2022 : Learning the Feedback Connections from V1 to LGN via Information Maximization »
Reza Eghbali · Fritz Sommer · Murray Sherman -
2022 : Panel Discussion II: Geometric and topological principles for representations in the brain »
Bruno Olshausen · Kristopher Jensen · Gabriel Kreiman · Manu Madhav · Christian A Shewmake -
2022 : In search of invariance in brains and machines »
Bruno Olshausen -
2022 Poster: Operative dimensions in unconstrained connectivity of recurrent neural networks »
Renate Krause · Matthew Cook · Sepp Kollmorgen · Valerio Mante · Giacomo Indiveri -
2019 Poster: Superposition of many models into one »
Brian Cheung · Alexander Terekhov · Yubei Chen · Pulkit Agrawal · Bruno Olshausen -
2018 Poster: The Sparse Manifold Transform »
Yubei Chen · Dylan Paiton · Bruno Olshausen -
2014 Poster: Information-based learning by agents in unbounded state spaces »
Shariq A Mobin · James A Arnemann · Fritz Sommer -
2013 Workshop: High-dimensional Statistical Inference in the Brain »
Alyson Fletcher · Dmitri B Chklovskii · Fritz Sommer · Ian H Stevenson -
2010 Spotlight: Deciphering subsampled data: adaptive compressive sampling as a principle of brain communication »
Guy D Isely · Christopher J Hillar · Fritz Sommer -
2010 Poster: Deciphering subsampled data: adaptive compressive sampling as a principle of brain communication »
Guy D Isely · Christopher J Hillar · Fritz Sommer -
2010 Poster: Group Sparse Coding with a Laplacian Scale Mixture Prior »
Pierre J Garrigues · Bruno Olshausen -
2009 Workshop: Manifolds, sparsity, and structured models: When can low-dimensional geometry really help? »
Richard Baraniuk · Volkan Cevher · Mark A Davenport · Piotr Indyk · Bruno Olshausen · Michael B Wakin -
2009 Poster: Learning transport operators for image manifolds »
Jack Culpepper · Bruno Olshausen -
2008 Poster: Learning Transformational Invariants from Time-Varying Natural Images »
Charles Cadieu · Bruno Olshausen -
2008 Spotlight: Learning Transformational Invariants from Time-Varying Natural Images »
Charles Cadieu · Bruno Olshausen -
2007 Poster: Learning Horizontal Connections in a Sparse Coding Model of Natural Images »
Pierre Garrigues · Bruno Olshausen