Timezone: »

 
Neuromorphic Visual Scene Understanding with Resonator Networks (in brief)
Alpha Renner · Giacomo Indiveri · Lazar Supic · Andreea Danielescu · Bruno Olshausen · Fritz Sommer · Yulia Sandamirskaya · Edward Frady

@
in Symmetry and Geometry in Neural Representations (NeurReps) »
Event URL: https://openreview.net/forum?id=zmyhHJzJ1kz »

Inferring the position of objects and their rigid transformations is still an open problem in visual scene understanding. Here we propose a neuromorphic framework that poses scene understanding as a factorization problem and uses a resonator network to extract object identities and their transformations. The framework uses vector binding operations to produce generative image models in which binding acts as the equivariant operation for geometric transformations. A scene can therefore be described as a sum of vector products, which in turn can be efficiently factorized by a resonator network to infer objects and their poses. We also describe a hierarchical resonator network that enables the definition of a partitioned architecture in which vector binding is equivariant for horizontal and vertical translation within one partition, and for rotation and scaling within the other partition. We demonstrate our approach using synthetic scenes composed of simple 2D shapes undergoing rigid geometric transformations and color changes.

Keywords: Equivariance · recurrent neural networks · generative model · transform invariance · disentangle

Author Information

Alpha Renner (Institute of Neuroinformatics, University of Zurich and ETH Zurich)
Giacomo Indiveri (University of Zurich and ETH Zurich)
Giacomo Indiveri

Giacomo Indiveri is a dual Professor at the Faculty of Science of the University of Zurich and at Department of Information Technology and Electrical Engineering of ETH Zurich, Switzerland. He is the director of the Institute of Neuroinformatics of the University of Zurich and ETH Zurich. He obtained an M.Sc. degree in electrical engineering in 1992 and a Ph.D. degree in computer science from the University of Genoa, Italy in 2004. Engineer by training, Indiveri has also expertise in neuroscience, computer science, and machine learning. He has been combining these disciplines by studying natural and artificial intelligence in neural processing systems and in neuromorphic cognitive agents. His latest research interests lie in the study of spike-based learning mechanisms and recurrent networks of biologically plausible neurons, and in their integration in real-time closed-loop sensory-motor systems designed using analog/digital circuits and emerging memory technologies. His group uses these neuromorphic circuits to validate brain inspired computational paradigms in real-world scenarios, and to develop a new generation of fault-tolerant event-based neuromorphic computing technologies. Indiveri is senior member of the IEEE society, and a recipient of the 2021 IEEE Biomedical Circuits and Systems Best Paper Award. He is also an ERC fellow, recipient of three European Research Council grants.

Lazar Supic (University of California Berkeley)
Andreea Danielescu (Accenture Labs)
Bruno Olshausen (Redwood Center/UC Berkeley)
Fritz Sommer (UC Berkeley)
Yulia Sandamirskaya (Intel)
Edward Frady (Intel)

More from the Same Authors