Timezone: »
Attention in psychology and neuroscience conceptualizes how the human mind prioritizes information as a result of limited resources. Machine learning systems do not necessarily share the same limits, but implementations of attention have nevertheless proven useful in machine learning across a broad set of domains. Why is this so? I will focus on one aspect: interpretability, which is an ongoing challenge for machine learning systems. I will discuss two different implementations of attention in machine learning that tie closely to conceptualizations of attention in two domains of psychological research. Using these case studies as a starting point, I will discuss the broader strengths and drawbacks of using attention to constrain and interpret how machine learning systems process information. I will end with a problem statement highlighting the need to move away from localized notions to a global view of how attention-like mechanisms modulate information processing in artificial systems.
Author Information
Erin Grant (University College London)
More from the Same Authors
-
2021 : Meta-learning inductive biases of learning systems with Gaussian processes »
Michael Li · Erin Grant · Tom Griffiths -
2022 : Panel »
Erin Grant · Richard Turner · Neil Houlsby · Priyanka Agrawal · Abhijeet Awasthi · Salomey Osei -
2021 Workshop: 5th Workshop on Meta-Learning »
Erin Grant · Fábio Ferreira · Frank Hutter · Jonathan Richard Schwarz · Joaquin Vanschoren · Huaxiu Yao -
2021 Oral: Passive attention in artificial neural networks predicts human visual selectivity »
Thomas Langlois · Haicheng Zhao · Erin Grant · Ishita Dasgupta · Tom Griffiths · Nori Jacoby -
2021 Poster: Passive attention in artificial neural networks predicts human visual selectivity »
Thomas Langlois · Haicheng Zhao · Erin Grant · Ishita Dasgupta · Tom Griffiths · Nori Jacoby -
2020 : Introduction for invited speaker, Kate Rakelly »
Erin Grant -
2020 : Introduction for invited speaker, Fei-Fei Li »
Erin Grant -
2020 Workshop: Meta-Learning »
Jane Wang · Joaquin Vanschoren · Erin Grant · Jonathan Richard Schwarz · Francesco Visin · Jeff Clune · Roberto Calandra -
2020 : Opening remarks from the WiML 2020 Organizers »
Xinyi Chen · Erin Grant -
2020 Affinity Workshop: Women in Machine Learning »
Xinyi Chen · Erin Grant · Kristy Choi · Krystal Maughan · Xenia Miscouridou · Judy Hanwen Shen · Raquel Aoki · Belén Saldías · Mel Woghiren · Elizabeth Wood -
2019 : Q&A from the Audience. Ask the Grad Students »
Erin Grant · Ruairidh Battleday · Sophia Sanborn · Nadine Chang · Nikhil Parthasarathy -
2019 : Taxonomic structure in learning from few positive examples »
Erin Grant -
2019 : Meta-learning as hierarchical modeling »
Erin Grant -
2019 Poster: Reconciling meta-learning and continual learning with online mixtures of tasks »
Ghassen Jerfel · Erin Grant · Tom Griffiths · Katherine Heller -
2019 Spotlight: Reconciling meta-learning and continual learning with online mixtures of tasks »
Ghassen Jerfel · Erin Grant · Tom Griffiths · Katherine Heller -
2018 Workshop: NIPS 2018 Workshop on Meta-Learning »
Joaquin Vanschoren · Frank Hutter · Sachin Ravi · Jane Wang · Erin Grant -
2017 : POSTER: Concept acquisition through meta-learning »
Erin Grant