Timezone: »
Developments in machine learning interpretability techniques over the past decade have provided new tools to observe the image regions that are most informative for classification and localization in artificial neural networks (ANNs). Are the same regions similarly informative to human observers? Using data from 79 new experiments and 7,810 participants, we show that passive attention techniques reveal a significant overlap with human visual selectivity estimates derived from 6 distinct behavioral tasks including visual discrimination, spatial localization, recognizability, free-viewing, cued-object search, and saliency search fixations. We find that input visualizations derived from relatively simple ANN architectures probed using guided backpropagation methods are the best predictors of a shared component in the joint variability of the human measures. We validate these correlational results with causal manipulations using recognition experiments. We show that images masked with ANN attention maps were easier for humans to classify than control masks in a speeded recognition experiment. Similarly, we find that recognition performance in the same ANN models was likewise influenced by masking input images using human visual selectivity maps. This work contributes a new approach to evaluating the biological and psychological validity of leading ANNs as models of human vision: by examining their similarities and differences in terms of their visual selectivity to the information contained in images.
Author Information
Thomas Langlois (Princeton University)
Haicheng Zhao (Princeton University)
Erin Grant (UC Berkeley)
Ishita Dasgupta (Harvard University)
Tom Griffiths (Princeton University)
Nori Jacoby (Max Planck Institute for Empirical Aesthetics)
I'm interested in exploring the role of culture in auditory perception, using iterated learning alongside classical psychophysical methods to characterize perceptual biases in music and speech rhythms in populations around the world. Other work has focused on the mathematical modeling of sensorimotor synchronization in the form of tapping experiments as well as the application of machine-learning techniques to model aspects of musical syntax, including tonal harmony, birdsong, and the perception of musical form. I am currently a Research Group Leader at the Max Planck Institute for Empirical Aesthetics in Frankfurt, where I direct the "Computational Auditory Perception" group. Previously, I was a Presidential Scholar In Society And Neuroscience at Columbia University, a postdoc at the McDermott Computational Audition Lab at MIT, and a visiting postdoctoral researcher in Tom Griffiths's Computational Cognitive Science Lab at Berkeley. I completed my Ph.D. at the Edmond and Lily Safra Center for Brain Sciences (ELSC) at the Hebrew University of Jerusalem under the supervision of Naftali Tishby and Merav Ahissar, and hold a M.A. in mathematics from the same institution. My research has been published in journals including Current Biology, Science, Nature, Nature Scientific Reports, Philosophical Transactions B, Journal of Neuroscience, Journal of Vision, and Psychonomic Bulletin and Review.
Related Events (a corresponding poster, oral, or spotlight)
-
2021 Poster: Passive attention in artificial neural networks predicts human visual selectivity »
Tue. Dec 7th 04:30 -- 06:00 PM Room
More from the Same Authors
-
2021 : Meta-learning inductive biases of learning systems with Gaussian processes »
Michael Li · Erin Grant · Tom Griffiths -
2022 : Hierarchical Abstraction for Combinatorial Generalization in Object Rearrangement »
Michael Chang · Alyssa L Dayan · Franziska Meier · Tom Griffiths · Sergey Levine · Amy Zhang -
2022 : Hierarchical Abstraction for Combinatorial Generalization in Object Rearrangement »
Michael Chang · Alyssa L Dayan · Franziska Meier · Tom Griffiths · Sergey Levine · Amy Zhang -
2022 : How to talk so AI will learn: instructions, descriptions, and pragmatics »
Theodore Sumers · Robert Hawkins · Mark Ho · Tom Griffiths · Dylan Hadfield-Menell -
2022 : Hierarchical Abstraction for Combinatorial Generalization in Object Rearrangement »
Michael Chang · Alyssa L Dayan · Franziska Meier · Tom Griffiths · Sergey Levine · Amy Zhang -
2022 : Hierarchical Abstraction for Combinatorial Generalization in Object Rearrangement »
Michael Chang · Alyssa L Dayan · Franziska Meier · Tom Griffiths · Sergey Levine · Amy Zhang -
2022 : Hierarchical Abstraction for Combinatorial Generalization in Object Rearrangement »
Michael Chang · Alyssa L Dayan · Franziska Meier · Tom Griffiths · Sergey Levine · Amy Zhang -
2022 : Hierarchical Abstraction for Combinatorial Generalization in Object Rearrangement »
Michael Chang · Alyssa L Dayan · Franziska Meier · Tom Griffiths · Sergey Levine · Amy Zhang -
2022 : Hierarchical Abstraction for Combinatorial Generalization in Object Rearrangement »
Michael Chang · Alyssa L Dayan · Franziska Meier · Tom Griffiths · Sergey Levine · Amy Zhang -
2022 : On the informativeness of supervision signals »
Ilia Sucholutsky · Raja Marjieh · Tom Griffiths -
2022 : Panel »
Erin Grant · Richard Turner · Neil Houlsby · Priyanka Agrawal · Abhijeet Awasthi · Salomey Osei -
2022 : On the informativeness of supervision signals »
Ilia Sucholutsky · Raja Marjieh · Tom Griffiths -
2022 : Hierarchical Abstraction for Combinatorial Generalization in Object Rearrangement »
Michael Chang · Alyssa L Dayan · Franziska Meier · Tom Griffiths · Sergey Levine · Amy Zhang -
2022 : Attention as Interpretable Information Processing in Machine Learning Systems »
Erin Grant -
2022 Workshop: Shared Visual Representations in Human and Machine Intelligence (SVRHM) »
Arturo Deza · Joshua Peterson · N Apurva Ratan Murty · Tom Griffiths -
2022 Poster: Using natural language and program abstractions to instill human inductive biases in machines »
Sreejan Kumar · Carlos G. Correa · Ishita Dasgupta · Raja Marjieh · Michael Y Hu · Robert Hawkins · Jonathan D Cohen · nathaniel daw · Karthik Narasimhan · Tom Griffiths -
2022 Poster: How to talk so AI will learn: Instructions, descriptions, and autonomy »
Theodore Sumers · Robert Hawkins · Mark Ho · Tom Griffiths · Dylan Hadfield-Menell -
2022 Poster: Object Representations as Fixed Points: Training Iterative Refinement Algorithms with Implicit Differentiation »
Michael Chang · Tom Griffiths · Sergey Levine -
2021 : Reinforcement learning: It's all in the mind »
Tom Griffiths -
2021 Workshop: Workshop on Human and Machine Decisions »
Daniel Reichman · Joshua Peterson · Kiran Tomlinson · Annie Liang · Tom Griffiths -
2021 : Opening remarks »
Tom Griffiths -
2021 : Exploring the Structure of Human Adjective Representations »
Karan Grewal · Joshua Peterson · Bill Thompson · Tom Griffiths -
2021 : Invited Talk 4 »
Tom Griffiths -
2021 Workshop: Shared Visual Representations in Human and Machine Intelligence »
Arturo Deza · Joshua Peterson · N Apurva Ratan Murty · Tom Griffiths -
2021 Workshop: 5th Workshop on Meta-Learning »
Erin Grant · Fábio Ferreira · Frank Hutter · Jonathan Richard Schwarz · Joaquin Vanschoren · Huaxiu Yao -
2020 : Invited Talk #5 QnA - Ishita Dasgupta »
Ishita Dasgupta · Julie J Lee · Feryal Behbahani · Raymond Chua -
2020 : Invited Talk #5 Ishita Dasgupta - Embedding structure in data: Progress and challenges for the meta-learning approach »
Ishita Dasgupta -
2020 : Panel Discussions »
Grace Lindsay · George Konidaris · Shakir Mohamed · Kimberly Stachenfeld · Peter Dayan · Yael Niv · Doina Precup · Catherine Hartley · Ishita Dasgupta -
2020 Workshop: Shared Visual Representations in Human and Machine Intelligence (SVRHM) »
Arturo Deza · Joshua Peterson · N Apurva Ratan Murty · Tom Griffiths -
2020 : Introduction for invited speaker, Kate Rakelly »
Erin Grant -
2020 : Introduction for invited speaker, Fei-Fei Li »
Erin Grant -
2020 Workshop: Meta-Learning »
Jane Wang · Joaquin Vanschoren · Erin Grant · Jonathan Richard Schwarz · Francesco Visin · Jeff Clune · Roberto Calandra -
2020 Poster: Gibbs Sampling with People »
Peter Harrison · Raja Marjieh · Federico G Adolfi · Pol van Rijn · Manuel Anglada-Tort · Ofer Tchernichovski · Pauline Larrouy-Maestri · Nori Jacoby -
2020 Oral: Gibbs Sampling with People »
Peter Harrison · Raja Marjieh · Federico G Adolfi · Pol van Rijn · Manuel Anglada-Tort · Ofer Tchernichovski · Pauline Larrouy-Maestri · Nori Jacoby -
2020 : Opening remarks from the WiML 2020 Organizers »
Xinyi Chen · Erin Grant -
2020 Affinity Workshop: Women in Machine Learning »
Xinyi Chen · Erin Grant · Kristy Choi · Krystal Maughan · Xenia Miscouridou · Judy Hanwen Shen · Raquel Aoki · Belén Saldías · Mel Woghiren · Elizabeth Wood -
2019 : Concluding Remarks & Prizes Ceremony »
Arturo Deza · Joshua Peterson · Apurva Ratan Murty · Tom Griffiths -
2019 : Tom Griffiths »
Tom Griffiths -
2019 : Coffee/Poster session 2 »
Xingyou Song · Puneet Mangla · David Salinas · Zhenxun Zhuang · Leo Feng · Shell Xu Hu · Raul Puri · Wesley Maddox · Aniruddh Raghu · Prudencio Tossou · Mingzhang Yin · Ishita Dasgupta · Kangwook Lee · Ferran Alet · Zhen Xu · Jörg Franke · James Harrison · Jonathan Warrell · Guneet Dhillon · Arber Zela · Xin Qiu · Julien Niklas Siems · Russell Mendonca · Louis Schlessinger · Jeffrey Li · Georgiana Manolache · Debojyoti Dutta · Lucas Glass · Abhishek Singh · Gregor Koehler -
2019 : Q&A from the Audience. Ask the Grad Students »
Erin Grant · Ruairidh Battleday · Sophia Sanborn · Nadine Chang · Nikhil Parthasarathy -
2019 : Taxonomic structure in learning from few positive examples »
Erin Grant -
2019 : Meta-learning as hierarchical modeling »
Erin Grant -
2019 : Opening Remarks »
Arturo Deza · Joshua Peterson · Apurva Ratan Murty · Tom Griffiths -
2019 Workshop: Shared Visual Representations in Human and Machine Intelligence »
Arturo Deza · Joshua Peterson · Apurva Ratan Murty · Tom Griffiths -
2019 Poster: Reconciling meta-learning and continual learning with online mixtures of tasks »
Ghassen Jerfel · Erin Grant · Tom Griffiths · Katherine Heller -
2019 Spotlight: Reconciling meta-learning and continual learning with online mixtures of tasks »
Ghassen Jerfel · Erin Grant · Tom Griffiths · Katherine Heller -
2019 Poster: On the Utility of Learning about Humans for Human-AI Coordination »
Micah Carroll · Rohin Shah · Mark Ho · Tom Griffiths · Sanjit Seshia · Pieter Abbeel · Anca Dragan -
2018 Workshop: NIPS 2018 Workshop on Meta-Learning »
Joaquin Vanschoren · Frank Hutter · Sachin Ravi · Jane Wang · Erin Grant -
2017 : POSTER: Concept acquisition through meta-learning »
Erin Grant