Timezone: »
When developing deep learning models, we usually decide what task we want to solve then search for a model that generalizes well on the task. An intriguing question would be: what if, instead of fixing the task and searching in the model space, we fix the model and search in the task space? Can we find tasks that the model generalizes on? How do they look, or do they indicate anything? These are the questions we address in this paper. We propose a task discovery framework that automatically finds examples of such tasks via optimizing a generalization-based quantity called agreement score. We demonstrate that one set of images can give rise to many tasks on which neural networks generalize well. These tasks are a reflection of the inductive biases of the learning framework and the statistical patterns present in the data, thus they can make a useful tool for analyzing the neural networks and their biases. As an example, we show that the discovered tasks can be used to automatically create ''adversarial train-test splits'' which make a model fail at test time, without changing the pixels or labels, but by only selecting how the datapoints should be split between the train and test sets. We end with a discussion on human-interpretability of the discovered tasks.
Author Information
Andrei Atanov (EPFL)
Andrei Filatov (Moscow Institute of Physics and Technology)
Teresa Yeo (EPFL)
Ajay Sohmshetty (Harmonic AI)
Amir Zamir (Swiss Federal Institute of Technology (EPFL))
More from the Same Authors
-
2023 Poster: 4M: Massively Multimodal Masked Modeling »
David Mizrahi · Roman Bachmann · Oguzhan Kar · Teresa Yeo · Mingfei Gao · Afshin Dehghan · Amir Zamir -
2022 Spotlight: Lightning Talks 1B-4 »
Andrei Atanov · Shiqi Yang · Wanshan Li · Yongchang Hao · Ziquan Liu · Jiaxin Shi · Anton Plaksin · Jiaxiang Chen · Ziqi Pan · yaxing wang · Yuxin Liu · Stepan Martyanov · Alessandro Rinaldo · Yuhao Zhou · Li Niu · Qingyuan Yang · Andrei Filatov · Yi Xu · Liqing Zhang · Lili Mou · Ruomin Huang · Teresa Yeo · kai wang · Daren Wang · Jessica Hwang · Yuanhong Xu · Qi Qian · Hu Ding · Michalis Titsias · Shangling Jui · Ajay Sohmshetty · Lester Mackey · Joost van de Weijer · Hao Li · Amir Zamir · Xiangyang Ji · Antoni Chan · Rong Jin -
2022 Spotlight: Task Discovery: Finding the Tasks that Neural Networks Generalize on »
Andrei Atanov · Andrei Filatov · Teresa Yeo · Ajay Sohmshetty · Amir Zamir -
2022 Poster: PALMER: Perception - Action Loop with Memory for Long-Horizon Planning »
Onur Beker · Mohammad Mohammadi · Amir Zamir