Timezone: »
Meta-learning extracts the common knowledge from learning different tasks and uses it for unseen tasks. It can significantly improve tasks that suffer from insufficient training data, e.g., few-shot learning. In most meta-learning methods, tasks are implicitly related by sharing parameters or optimizer. In this paper, we show that a meta-learner that explicitly relates tasks on a graph describing the relations of their output dimensions (e.g., classes) can significantly improve few-shot learning. The graph’s structure is usually free or cheap to obtain but has rarely been explored in previous works. We develop a novel meta-learner of this type for prototype based classification, in which a prototype is generated for each class, such that the nearest neighbor search among the prototypes produces an accurate classification. The meta-learner, called “Gated Propagation Network (GPN)”, learns to propagate messages between prototypes of different classes on the graph, so that learning the prototype of each class benefits from the data of other related classes. In GPN, an attention mechanism aggregates messages from neighboring classes of each class, with a gate choosing between the aggregated message and the message from the class itself. We train GPN on a sequence of tasks from many-shot to few-shot generated by subgraph sampling. During training, it is able to reuse and update previously achieved prototypes from the memory in a life-long learning cycle. In experiments, under different training-test discrepancy and test task generation settings, GPN outperforms recent meta-learning methods on two benchmark datasets. Code of GPN is publicly available at: https://github.com/liulu112601/Gated-Propagation-Net.
Author Information
LU LIU (University of Technology Sydney)
Lu Liu is a 3-rd year Ph.D. student from University of Technology Sydney. Her research interests lie in Machine Learning, Meta-learning and Low-shot learning.
Tianyi Zhou (University of Washington, Seattle)

Tianyi Zhou (https://tianyizhou.github.io) is a tenure-track assistant professor of computer science at the University of Maryland, College Park. He received his Ph.D. from the school of computer science & engineering at the University of Washington, Seattle. His research interests are in machine learning, optimization, and natural language processing (NLP). His recent works study curriculum learning that can combine high-level human learning strategies with model training dynamics to create a hybrid intelligence. The applications include semi/self-supervised learning, robust learning, reinforcement learning, meta-learning, ensemble learning, etc. He published >80 papers and is a recipient of the Best Student Paper Award at ICDM 2013 and the 2020 IEEE Computer Society TCSC Most Influential Paper Award.
Guodong Long (University of Technology Sydney (UTS))
Jing Jiang (University of Technology Sydney)
Chengqi Zhang (University of Technology Sydney)
More from the Same Authors
-
2021 Spotlight: Constrained Robust Submodular Partitioning »
Shengjie Wang · Tianyi Zhou · Chandrashekhar Lavania · Jeff A Bilmes -
2023 Poster: Is Heterogeneity Notorious? Taming Heterogeneity to Handle Test-Time Shift in Federated Learning »
Yue Tan · Chen Chen · Weiming Zhuang · Xin Dong · Lingjuan Lyu · Guodong Long -
2023 Poster: Unsupervised Video Domain Adaptation for Action Recognition: A Disentanglement Perspective »
Pengfei Wei · Lingdong Kong · Xinghua Qu · Yi Ren · Zhiqiang Xu · Jing Jiang · Xiang Yin -
2023 Poster: Structured Federated Learning through Clustered Additive Modeling »
Jie Ma · Tianyi Zhou · Guodong Long · Jing Jiang · Chengqi Zhang -
2022 Spotlight: Federated Learning from Pre-Trained Models: A Contrastive Learning Approach »
Yue Tan · Guodong Long · Jie Ma · LU LIU · Tianyi Zhou · Jing Jiang -
2022 Spotlight: Lightning Talks 3A-1 »
Shu Ding · Wanxing Chang · Jiyang Guan · Mouxiang Chen · Guan Gui · Yue Tan · Shiyun Lin · Guodong Long · Yuze Han · Wei Wang · Zhen Zhao · Ye Shi · Jian Liang · Chenghao Liu · Lei Qi · Ran He · Jie Ma · Zemin Liu · Xiang Li · Hoang Tuan · Luping Zhou · Zhihua Zhang · Jianling Sun · Jingya Wang · LU LIU · Tianyi Zhou · Lei Wang · Jing Jiang · Yinghuan Shi -
2022 Spotlight: Adversarial Auto-Augment with Label Preservation: A Representation Learning Principle Guided Approach »
Kaiwen Yang · Yanchao Sun · Jiahao Su · Fengxiang He · Xinmei Tian · Furong Huang · Tianyi Zhou · Dacheng Tao -
2022 Poster: Federated Learning from Pre-Trained Models: A Contrastive Learning Approach »
Yue Tan · Guodong Long · Jie Ma · LU LIU · Tianyi Zhou · Jing Jiang -
2022 Poster: Adversarial Auto-Augment with Label Preservation: A Representation Learning Principle Guided Approach »
Kaiwen Yang · Yanchao Sun · Jiahao Su · Fengxiang He · Xinmei Tian · Furong Huang · Tianyi Zhou · Dacheng Tao -
2022 Poster: Retrospective Adversarial Replay for Continual Learning »
Lilly Kumari · Shengjie Wang · Tianyi Zhou · Jeff A Bilmes -
2021 Poster: Constrained Robust Submodular Partitioning »
Shengjie Wang · Tianyi Zhou · Chandrashekhar Lavania · Jeff A Bilmes -
2021 Poster: Class-Disentanglement and Applications in Adversarial Detection and Defense »
Kaiwen Yang · Tianyi Zhou · Yonggang Zhang · Xinmei Tian · Dacheng Tao -
2021 Poster: CO-PILOT: COllaborative Planning and reInforcement Learning On sub-Task curriculum »
Shuang Ao · Tianyi Zhou · Guodong Long · Qinghua Lu · Liming Zhu · Jing Jiang -
2021 Poster: Recognizing Vector Graphics without Rasterization »
XINYANG JIANG · LU LIU · Caihua Shan · Yifei Shen · Xuanyi Dong · Dongsheng Li -
2020 Poster: Curriculum Learning by Dynamic Instance Hardness »
Tianyi Zhou · Shengjie Wang · Jeffrey A Bilmes -
2020 Poster: MESA: Boost Ensemble Imbalanced Learning with MEta-SAmpler »
Zhining Liu · Pengfei Wei · Jing Jiang · Wei Cao · Jiang Bian · Yi Chang -
2020 Poster: Deep Reinforcement Learning with Stacked Hierarchical Attention for Text-based Games »
Yunqiu Xu · Meng Fang · Ling Chen · Yali Du · Joey Tianyi Zhou · Chengqi Zhang -
2020 Poster: Cooperative Heterogeneous Deep Reinforcement Learning »
Han Zheng · Pengfei Wei · Jing Jiang · Guodong Long · Qinghua Lu · Chengqi Zhang -
2019 Poster: Curriculum-guided Hindsight Experience Replay »
Meng Fang · Tianyi Zhou · Yali Du · Lei Han · Zhengyou Zhang -
2018 Poster: Diverse Ensemble Evolution: Curriculum Data-Model Marriage »
Tianyi Zhou · Shengjie Wang · Jeffrey A Bilmes -
2014 Poster: Divide-and-Conquer Learning by Anchoring a Conical Hull »
Tianyi Zhou · Jeffrey A Bilmes · Carlos Guestrin