Do Deep Neural Networks Suffer from Crowding?
Anna Volokitin · Gemma Roig · Tomaso Poggio

Mon Dec 4th 06:30 -- 10:30 PM @ Pacific Ballroom #10 #None

Crowding is a visual effect suffered by humans, in which an object that can be recognized in isolation can no longer be recognized when other objects, called flankers, are placed close to it. In this work, we study the effect of crowding in artificial Deep Neural Networks (DNNs) for object recognition. We analyze both deep convolutional neural networks (DCNNs) as well as an extension of DCNNs that are multi-scale and that change the receptive field size of the convolution filters with their position in the image. The latter networks, that we call eccentricity-dependent, have been proposed for modeling the feedforward path of the primate visual cortex. Our results reveal that the eccentricity-dependent model, trained on target objects in isolation, can recognize such targets in the presence of flankers, if the targets are near the center of the image, whereas DCNNs cannot. Also, for all tested networks, when trained on targets in isolation, we find that recognition accuracy of the networks decreases the closer the flankers are to the target and the more flankers there are. We find that visual similarity between the target and flankers also plays a role and that pooling in early layers of the network leads to more crowding. Additionally, we show that incorporating flankers into the images of the training set for learning the DNNs does not lead to robustness against configurations not seen at training.

Author Information

Anna Volokitin (ETH Zurich)
Gemma Roig (SUTD)
Tomaso Poggio (MIT)

Tomaso A. Poggio, is the Eugene McDermott Professor in the Dept. of Brain & Cognitive Sciences at MIT and the director of the new NSF Center for Brains, Minds and Machines at MIT of which MIT and Harvard are the main member Institutions. He is a member of both the Computer Science and Artificial Intelligence Laboratory and of the McGovern Brain Institute. He is an honorary member of the Neuroscience Research Program, a member of the American Academy of Arts and Sciences, a Founding Fellow of AAAI and a founding member of the McGovern Institute for Brain Research. Among other honors he received the Laurea Honoris Causa from the University of Pavia for the Volta Bicentennial, the 2003 Gabor Award, the Okawa Prize 2009, the AAAS Fellowship and the 2014 Swartz Prize for Theoretical and Computational Neuroscience. He is one of the most cited computational scientists with contributions ranging from the biophysical and behavioral studies of the visual system to the computational analyses of vision and learning in humans and machines. With W. Reichardt he characterized quantitatively the visuo-motor control system in the fly. With D. Marr, he introduced the seminal idea of levels of analysis in computational neuroscience. He introduced regularization as a mathematical framework to approach the ill-posed problems of vision and the key problem of learning from data. In the last decade he has developed an influential hierarchical model of visual recognition in the visual cortex. The citation for the recent 2009 Okawa prize mentions his “…outstanding contributions to the establishment of computational neuroscience, and pioneering researches ranging from the biophysical and behavioral studies of the visual system to the computational analysis of vision and learning in humans and machines.” His research has always been interdisciplinary, between brains and computers. It is now focused on the mathematics of learning theory, the applications of learning techniques to computer vision and especially on computational neuroscience of the visual cortex. A former Corporate Fellow of Thinking Machines Corporation and a former director of PHZ Capital Partners, Inc., he is a director of Mobileye and was involved in starting, or investing in, several other high tech companies including Arris Pharmaceutical, nFX, Imagen, Digital Persona and Deep Mind. Tomaso Poggio Eugene McDermott Professor Director NSF Science & Technology Center for Brains, Minds and Machines(CBMM) Core founding scientific advisor, MIT Quest for Intelligence McGovern Institute CSAIL (Computer Science and Artificial Intelligence Lab) Brain Sciences Department M.I.T., 46-5177B see 43 Vassar Street Cambridge, MA 02142 E-mail: Phone: 617-253-5230 Fax: 617-253-2964 Web: PoggioLab Web page:

More from the Same Authors