Timezone: »

 
Poster
MATE: Plugging in Model Awareness to Task Embedding for Meta Learning
Xiaohan Chen · Zhangyang Wang · Siyu Tang · Krikamol Muandet

Wed Dec 09 09:00 PM -- 11:00 PM (PST) @ Poster Session 4 #1205
in Poster Session 5 »

Meta-learning improves generalization of machine learning models when faced with previously unseen tasks by leveraging experiences from different, yet related prior tasks. To allow for better generalization, we propose a novel task representation called model-aware task embedding (MATE) that incorporates not only the data distributions of different tasks, but also the complexity of the tasks through the models used. The task complexity is taken into account by a novel variant of kernel mean embedding, combined with an instance-adaptive attention mechanism inspired by an SVM-based feature selection algorithm. Together with conditioning layers in deep neural networks, MATE can be easily incorporated into existing meta learners as a plug-and-play module. While MATE is widely applicable to general tasks where the concept of task/environment is involved, we demonstrate its effectiveness in few-shot learning by improving a state-of-the-art model consistently on two benchmarks. Source codes for this paper are available at https://github.com/VITA-Group/MATE.

Author Information

Xiaohan Chen (University of Texas at Austin)
Zhangyang Wang (University of Texas at Austin)
Siyu Tang (ETH Zurich)
Krikamol Muandet (Max Planck Institute for Intelligent Systems)

More from the Same Authors