Timezone: »
Despite impressive performance on numerous visual tasks, Convolutional Neural Networks (CNNs) --- unlike brains --- are often highly sensitive to small perturbations of their input, e.g. adversarial noise leading to erroneous decisions. We propose to regularize CNNs using large-scale neuroscience data to learn more robust neural features in terms of representational similarity. We presented natural images to mice and measured the responses of thousands of neurons from cortical visual areas. Next, we denoised the notoriously variable neural activity using strong predictive models trained on this large corpus of responses from the mouse visual system, and calculated the representational similarity for millions of pairs of images from the model's predictions. We then used the neural representation similarity to regularize CNNs trained on image classification by penalizing intermediate representations that deviated from neural ones. This preserved performance of baseline models when classifying images under standard benchmarks, while maintaining substantially higher performance compared to baseline or control models when classifying noisy images. Moreover, the models regularized with cortical representations also improved model robustness in terms of adversarial attacks. This demonstrates that regularizing with neural data can be an effective tool to create an inductive bias towards more robust inference.
Author Information
Zhe Li (Baylor College of Medicine)
Wieland Brendel (AG Bethge, University of Tübingen)
Edgar Walker (Baylor College of Medicine)
Erick Cobos (Baylor College of Medicine)
Taliah Muhammad (Baylor College of Medicine)
Jacob Reimer (Baylor College of Medicine)
Matthias Bethge (University of Tübingen)
Fabian Sinz (University Tübingen)
Xaq Pitkow (BCM/Rice)
Andreas Tolias (Baylor College of Medicine)
More from the Same Authors
-
2020 Poster: System Identification with Biophysical Constraints: A Circuit Model of the Inner Retina »
Cornelius Schröder · David Klindt · Sarah Strauss · Katrin Franke · Matthias Bethge · Thomas Euler · Philipp Berens -
2020 Poster: Factorized Neural Processes for Neural Processes: K-Shot Prediction of Neural Responses »
Ronald (James) Cotton · Fabian Sinz · Andreas Tolias -
2020 Spotlight: System Identification with Biophysical Constraints: A Circuit Model of the Inner Retina »
Cornelius Schröder · David Klindt · Sarah Strauss · Katrin Franke · Matthias Bethge · Thomas Euler · Philipp Berens -
2020 Poster: Improving robustness against common corruptions by covariate shift adaptation »
Steffen Schneider · Evgenia Rusak · Luisa Eck · Oliver Bringmann · Wieland Brendel · Matthias Bethge -
2020 Poster: Inverse Rational Control with Partially Observable Continuous Nonlinear Dynamics »
Minhae Kwon · Saurabh Daptardar · Paul R Schrater · Xaq Pitkow -
2019 Poster: Accurate, reliable and fast robustness evaluation »
Wieland Brendel · Jonas Rauber · Matthias Kümmerer · Ivan Ustyuzhaninov · Matthias Bethge -
2018 Poster: Generalisation in humans and deep neural networks »
Robert Geirhos · Carlos R. M. Temme · Jonas Rauber · Heiko H. Schütt · Matthias Bethge · Felix A. Wichmann -
2018 Poster: Stimulus domain transfer in recurrent models for large scale cortical population prediction on video »
Fabian Sinz · Alexander Ecker · Paul Fahey · Edgar Walker · Erick Cobos · Emmanouil Froudarakis · Dimitri Yatsenko · Xaq Pitkow · Jacob Reimer · Andreas Tolias -
2017 Poster: Neural system identification for large populations separating “what” and “where” »
David Klindt · Alexander Ecker · Thomas Euler · Matthias Bethge -
2016 Poster: Inference by Reparameterization in Neural Population Codes »
Rajkumar Vasudeva Raju · Xaq Pitkow -
2015 Poster: Texture Synthesis Using Convolutional Neural Networks »
Leon A Gatys · Alexander Ecker · Matthias Bethge -
2015 Poster: Generative Image Modeling Using Spatial LSTMs »
Lucas Theis · Matthias Bethge -
2012 Poster: Training sparse natural image models with a fast Gibbs sampler of an extended state space »
Lucas Theis · Jascha Sohl-Dickstein · Matthias Bethge -
2010 Poster: Evaluating neuronal codes for inference using Fisher information »
Ralf Haefner · Matthias Bethge -
2009 Poster: Hierarchical Modeling of Local Image Features through $L_p$-Nested Symmetric Distributions »
Fabian H Sinz · Eero Simoncelli · Matthias Bethge -
2009 Poster: Neurometric function analysis of population codes »
Philipp Berens · Sebastian Gerwinn · Alexander S Ecker · Matthias Bethge -
2009 Poster: A joint maximum-entropy model for binary neural population patterns and continuous signals »
Sebastian Gerwinn · Philipp Berens · Matthias Bethge -
2009 Spotlight: A joint maximum-entropy model for binary neural population patterns and continuous signals »
Sebastian Gerwinn · Philipp Berens · Matthias Bethge -
2009 Poster: Bayesian estimation of orientation preference maps »
Jakob H Macke · Sebastian Gerwinn · Leonard White · Matthias Kaschube · Matthias Bethge -
2008 Poster: The Conjoint Effect of Divisive Normalization and Orientation Selectivity on Redundancy Reduction »
Fabian H Sinz · Matthias Bethge -
2008 Spotlight: The Conjoint Effect of Divisive Normalization and Orientation Selectivity on Redundancy Reduction »
Fabian H Sinz · Matthias Bethge -
2007 Oral: Bayesian Inference for Spiking Neuron Models with a Sparsity Prior »
Sebastian Gerwinn · Jakob H Macke · Matthias Seeger · Matthias Bethge -
2007 Spotlight: Near-Maximum Entropy Models for Binary Neural Representations of Natural Images »
Matthias Bethge · Philipp Berens -
2007 Poster: Near-Maximum Entropy Models for Binary Neural Representations of Natural Images »
Matthias Bethge · Philipp Berens -
2007 Poster: Bayesian Inference for Spiking Neuron Models with a Sparsity Prior »
Sebastian Gerwinn · Jakob H Macke · Matthias Seeger · Matthias Bethge -
2007 Poster: Receptive Fields without Spike-Triggering »
Jakob H Macke · Günther Zeck · Matthias Bethge