Timezone: »

 
Poster
Meta-learning with an Adaptive Task Scheduler
Huaxiu Yao · Yu Wang · Ying Wei · Peilin Zhao · Mehrdad Mahdavi · Defu Lian · Chelsea Finn

Tue Dec 07 04:30 PM -- 06:00 PM (PST) @
in Poster Session 2 »

To benefit the learning of a new task, meta-learning has been proposed to transfer a well-generalized meta-model learned from various meta-training tasks. Existing meta-learning algorithms randomly sample meta-training tasks with a uniform probability, under the assumption that tasks are of equal importance. However, it is likely that tasks are detrimental with noise or imbalanced given a limited number of meta-training tasks. To prevent the meta-model from being corrupted by such detrimental tasks or dominated by tasks in the majority, in this paper, we propose an adaptive task scheduler (ATS) for the meta-training process. In ATS, for the first time, we design a neural scheduler to decide which meta-training tasks to use next by predicting the probability being sampled for each candidate task, and train the scheduler to optimize the generalization capacity of the meta-model to unseen tasks. We identify two meta-model-related factors as the input of the neural scheduler, which characterize the difficulty of a candidate task to the meta-model. Theoretically, we show that a scheduler taking the two factors into account improves the meta-training loss and also the optimization landscape. Under the setting of meta-learning with noise and limited budgets, ATS improves the performance on both miniImageNet and a real-world drug discovery benchmark by up to 13% and 18%, respectively, compared to state-of-the-art task schedulers.

Keywords: Optimization · Meta Learning

Author Information

Huaxiu Yao (Stanford University)
Yu Wang (University of Science and Technology of China)
Ying Wei (City University of Hong Kong)
Peilin Zhao (Tencent AI Lab)
Mehrdad Mahdavi (TTI Chicago)
Defu Lian (University of Science and Technology of China)
Chelsea Finn (Stanford)

More from the Same Authors