`

Timezone: »

 
Poster
Variational Continual Bayesian Meta-Learning
Qiang Zhang · Jinyuan Fang · Zaiqiao Meng · Shangsong Liang · Emine Yilmaz

Thu Dec 09 12:30 AM -- 02:00 AM (PST) @ None #None

Conventional meta-learning considers a set of tasks from a stationary distribution. In contrast, this paper focuses on a more complex online setting, where tasks arrive sequentially and follow a non-stationary distribution. Accordingly, we propose a Variational Continual Bayesian Meta-Learning (VC-BML) algorithm. VC-BML maintains a Dynamic Gaussian Mixture Model for meta-parameters, with the number of component distributions determined by a Chinese Restaurant Process. Dynamic mixtures at the meta-parameter level increase the capability to adapt to diverse tasks due to a larger parameter space, alleviating the negative knowledge transfer problem. To infer posteriors of model parameters, compared to the previously used point estimation method, we develop a more robust posterior approximation method -- structured variational inference for the sake of avoiding forgetting knowledge. Experiments on tasks from non-stationary distributions show that VC-BML is superior in transferring knowledge among diverse tasks and alleviating catastrophic forgetting in an online setting.

Author Information

Qiang Zhang (University College London)
Jinyuan Fang (Sun Yat-sen University)
Zaiqiao Meng (University of Glasgow)
Shangsong Liang (Sun Yat-sen University)
Emine Yilmaz (University College London)

More from the Same Authors

  • 2021 : Just Mix Once: Mixing Samples with Implicit Group Distribution »
    Giorgio Giannone · Serhii Havrylov · Jordan Massiah · Emine Yilmaz · Yunlong Jiao
  • 2021 Poster: Structure-Aware Random Fourier Kernel for Graphs »
    Jinyuan Fang · Qiang Zhang · Zaiqiao Meng · Shangsong Liang
  • 2020 Poster: Replica-Exchange Nos\'e-Hoover Dynamics for Bayesian Learning on Large Datasets »
    Rui Luo · Qiang Zhang · Yaodong Yang · Jun Wang
  • 2019 Poster: Semi-supervisedly Co-embedding Attributed Networks »
    Zaiqiao Meng · Shangsong Liang · Jinyuan Fang · Teng Xiao
  • 2018 : Poster Session 1 »
    Stefan Gadatsch · Danil Kuzin · Navneet Kumar · Patrick Dallaire · Tom Ryder · Remus-Petru Pop · Nathan Hunt · Adam Kortylewski · Sophie Burkhardt · Mahmoud Elnaggar · John Lawson · Yifeng Li · Jongha (Jon) Ryu · Juhan Bae · Micha Livne · Tim Pearce · Mariia Vladimirova · Jason E. Ramapuram · Jiaming Zeng · Xinyu Hu · Jiawei He · Danielle Maddix · Arunesh Mittal · Albert Shaw · Tuan Anh Le · Alexander Sagel · Lisha Chen · Victor Gallego · Mahdi Karami · Zihao Zhang · Tal Kachman · Noah Weber · Matt Benatan · Kumar K Sricharan · Vincent Cartillier · Ivan Ovinnikov · Buu Phan · Mahmoud Hossam · Liu Ziyin · Valerii Kharitonov · Eugene Golikov · Qiang Zhang · Jae Myung Kim · Sebastian Farquhar · Jishnu Mukhoti · Xu Hu · Gregory Gundersen · Lavanya Sita Tekumalla · Paris Perdikaris · Ershad Banijamali · Siddhartha Jain · Ge Liu · Martin Gottwald · Katy Blumer · Sukmin Yun · Ranganath Krishnan · Roman Novak · Yilun Du · Yu Gong · Beliz Gokkaya · Jessica Ai · Daniel Duckworth · Johannes von Oswald · Christian Henning · Louis-Philippe Morency · Ali Ghodsi · Mahesh Subedar · Jean-Pascal Pfister · RĂ©mi Lebret · Chao Ma · Aleksander Wieczorek · Laurence Perreault Levasseur