Timezone: »

Multi-Objective Meta Learning
Feiyang YE · Baijiong Lin · Zhixiong Yue · Pengxin Guo · Qiao Xiao · Yu Zhang

Tue Dec 07 08:30 AM -- 10:00 AM (PST) @ None #None

Meta learning with multiple objectives has been attracted much attention recently since many applications need to consider multiple factors when designing learning models. Existing gradient-based works on meta learning with multiple objectives mainly combine multiple objectives into a single objective in a weighted sum manner. This simple strategy usually works but it requires to tune the weights associated with all the objectives, which could be time consuming. Different from those works, in this paper, we propose a gradient-based Multi-Objective Meta Learning (MOML) framework without manually tuning weights. Specifically, MOML formulates the objective function of meta learning with multiple objectives as a Multi-Objective Bi-Level optimization Problem (MOBLP) where the upper-level subproblem is to solve several possibly conflicting objectives for the meta learner. To solve the MOBLP, we devise the first gradient-based optimization algorithm by alternatively solving the lower-level and upper-level subproblems via the gradient descent method and the gradient-based multi-objective optimization method, respectively. Theoretically, we prove the convergence properties of the proposed gradient-based optimization algorithm. Empirically, we show the effectiveness of the proposed MOML framework in several meta learning problems, including few-shot learning, domain adaptation, multi-task learning, and neural architecture search. The source code of MOML is available at https://github.com/Baijiong-Lin/MOML.

Author Information

Feiyang YE (University of Technology Sydney)
Baijiong Lin (Southern University of Science and Technology)
Zhixiong Yue (University of Technology Sydney)
Pengxin Guo (Southern University of Science and Technology)
Qiao Xiao (Southern University of Science and Technology)
Yu Zhang (HKUST)

More from the Same Authors