Timezone: »
Subpopulation shift widely exists in many real-world machine learning applications, referring to the training and test distributions containing the same subpopulation groups but varying in subpopulation frequencies. Importance reweighting is a normal way to handle the subpopulation shift issue by imposing constant or adaptive sampling weights on each sample in the training dataset. However, some recent studies have recognized that most of these approaches fail to improve the performance over empirical risk minimization especially when applied to over-parameterized neural networks. In this work, we propose a simple yet practical framework, called uncertainty-aware mixup (UMIX), to mitigate the overfitting issue in over-parameterized models by reweighting the ''mixed'' samples according to the sample uncertainty. The training-trajectories-based uncertainty estimation is equipped in the proposed UMIX for each sample to flexibly characterize the subpopulation distribution. We also provide insightful theoretical analysis to verify that UMIX achieves better generalization bounds over prior works. Further, we conduct extensive empirical studies across a wide range of tasks to validate the effectiveness of our method both qualitatively and quantitatively. Code is available at https://github.com/TencentAILabHealthcare/UMIX.
Author Information
Zongbo Han (Tianjin University)
Zhipeng Liang (Hong Kong University of Science and Technology)
Fan Yang (Tsinghua University, Tsinghua University)
Liu Liu (Tencent AI Lab)
Lanqing Li (Tencent AI Lab)
Yatao Bian (Tencent AI Lab)
Peilin Zhao (Tencent AI Lab)
Bingzhe Wu (Peeking University)
Changqing Zhang (Tianjin University)
Jianhua Yao (National Institutes of Health)
More from the Same Authors
-
2022 Poster: Learning Causally Invariant Representations for Out-of-Distribution Generalization on Graphs »
Yongqiang Chen · Yonggang Zhang · Yatao Bian · Han Yang · MA Kaili · Binghui Xie · Tongliang Liu · Bo Han · James Cheng -
2022 : Fast and Accurate Antibody Structure Prediction without Sequence Homologs »
Jiaxiang Wu · Fandi Wu · Biaobin Jiang · Wei Liu · Peilin Zhao -
2022 : Diversity Boosted Learning for Domain Generalization with A Large Number of Domains »
XI LENG · Yatao Bian · Xiaoying Tang -
2023 Poster: Learning Invariant Molecular Representation in Latent Discrete Space »
Xiang Zhuang · Qiang Zhang · Keyan Ding · Yatao Bian · Xiao Wang · Jingsong Lv · Hongyang Chen · Huajun Chen -
2023 Poster: Understanding and Improving Feature Learning for Out-of-Distribution Generalization »
Yongqiang Chen · Wei Huang · Kaiwen Zhou · Yatao Bian · Bo Han · James Cheng -
2023 Poster: Simplifying and Empowering Transformers for Large-Graph Representations »
Qitian Wu · Wentao Zhao · Chenxiao Yang · Hengrui Zhang · Fan Nie · Haitian Jiang · Yatao Bian · Junchi Yan -
2023 Poster: Does Invariant Graph Learning via Environment Augmentation Learn Invariance? »
Yongqiang Chen · Yatao Bian · Kaiwen Zhou · Binghui Xie · Bo Han · James Cheng -
2023 Poster: Fairness-guided Few-shot Prompting for Large Language Models »
Huan Ma · Changqing Zhang · Yatao Bian · Lemao Liu · Zhirui Zhang · Peilin Zhao · Shu Zhang · Huazhu Fu · Qinghua Hu · Bingzhe Wu -
2023 Workshop: Generative AI and Biology (GenBio@NeurIPS2023) »
Minkai Xu · Regina Barzilay · Jure Leskovec · Wenxian Shi · Menghua Wu · Zhenqiao Song · Lei Li · Fan Yang · Stefano Ermon -
2022 Spotlight: Lightning Talks 5B-3 »
Yanze Wu · Jie Xiao · Nianzu Yang · Jieyi Bi · Jian Yao · Yiting Chen · Qizhou Wang · Yangru Huang · Yongqiang Chen · Peixi Peng · Yuxin Hong · Xintao Wang · Feng Liu · Yining Ma · Qibing Ren · Xueyang Fu · Yonggang Zhang · Kaipeng Zeng · Jiahai Wang · GEN LI · Yonggang Zhang · Qitian Wu · Yifan Zhao · Chiyu Wang · Junchi Yan · Feng Wu · Yatao Bian · Xiaosong Jia · Ying Shan · Zhiguang Cao · Zheng-Jun Zha · Guangyao Chen · Tianjun Xiao · Han Yang · Jing Zhang · Jinbiao Chen · MA Kaili · Yonghong Tian · Junchi Yan · Chen Gong · Tong He · Binghui Xie · Yuan Sun · Francesco Locatello · Tongliang Liu · Yeow Meng Chee · David P Wipf · Tongliang Liu · Bo Han · Bo Han · Yanwei Fu · James Cheng · Zheng Zhang -
2022 Spotlight: Learning Causally Invariant Representations for Out-of-Distribution Generalization on Graphs »
Yongqiang Chen · Yonggang Zhang · Yatao Bian · Han Yang · MA Kaili · Binghui Xie · Tongliang Liu · Bo Han · James Cheng -
2022 Spotlight: Lightning Talks 2B-3 »
Jie-Jing Shao · Jiangmeng Li · Jiashuo Liu · Zongbo Han · Tianyang Hu · Jiayun Wu · Wenwen Qiang · Jun WANG · Zhipeng Liang · Lan-Zhe Guo · Wenjia Wang · Yanan Zhang · Xiao-wen Yang · Fan Yang · Bo Li · Wenyi Mo · Zhenguo Li · Liu Liu · Peng Cui · Yu-Feng Li · Changwen Zheng · Lanqing Li · Yatao Bian · Bing Su · Hui Xiong · Peilin Zhao · Bingzhe Wu · Changqing Zhang · Jianhua Yao -
2022 Spotlight: UMIX: Improving Importance Weighting for Subpopulation Shift via Uncertainty-Aware Mixup »
Zongbo Han · Zhipeng Liang · Fan Yang · Liu Liu · Lanqing Li · Yatao Bian · Peilin Zhao · Bingzhe Wu · Changqing Zhang · Jianhua Yao -
2022 Panel: Panel 1C-1: Learning Neural Set… & Holomorphic Equilibrium Propagation… »
Axel Laborieux · Yatao Bian -
2022 Poster: Learning Neural Set Functions Under the Optimal Subset Oracle »
Zijing Ou · Tingyang Xu · Qinliang Su · Yingzhen Li · Peilin Zhao · Yatao Bian -
2021 Poster: Generalized Linear Bandits with Local Differential Privacy »
Yuxuan Han · Zhipeng Liang · Yang Wang · Jiheng Zhang -
2021 Poster: Trustworthy Multimodal Regression with Mixture of Normal-inverse Gamma Distributions »
Huan Ma · Zongbo Han · Changqing Zhang · Huazhu Fu · Joey Tianyi Zhou · Qinghua Hu -
2021 Poster: Not All Low-Pass Filters are Robust in Graph Convolutional Networks »
Heng Chang · Yu Rong · Tingyang Xu · Yatao Bian · Shiji Zhou · Xin Wang · Junzhou Huang · Wenwu Zhu -
2021 Poster: Meta-learning with an Adaptive Task Scheduler »
Huaxiu Yao · Yu Wang · Ying Wei · Peilin Zhao · Mehrdad Mahdavi · Defu Lian · Chelsea Finn -
2020 Poster: RetroXpert: Decompose Retrosynthesis Prediction Like A Chemist »
Chaochao Yan · Qianggang Ding · Peilin Zhao · Shuangjia Zheng · JINYU YANG · Yang Yu · Junzhou Huang -
2020 Spotlight: RetroXpert: Decompose Retrosynthesis Prediction Like A Chemist »
Chaochao Yan · Qianggang Ding · Peilin Zhao · Shuangjia Zheng · JINYU YANG · Yang Yu · Junzhou Huang -
2020 Poster: Adversarial Sparse Transformer for Time Series Forecasting »
Sifan Wu · Xi Xiao · Qianggang Ding · Peilin Zhao · Ying Wei · Junzhou Huang -
2019 Poster: CPM-Nets: Cross Partial Multi-View Networks »
Changqing Zhang · Zongbo Han · yajie cui · Huazhu Fu · Joey Tianyi Zhou · Qinghua Hu -
2019 Spotlight: CPM-Nets: Cross Partial Multi-View Networks »
Changqing Zhang · Zongbo Han · yajie cui · Huazhu Fu · Joey Tianyi Zhou · Qinghua Hu -
2019 Poster: Hyperparameter Learning via Distributional Transfer »
Ho Chung Law · Peilin Zhao · Leung Sing Chan · Junzhou Huang · Dino Sejdinovic -
2019 Poster: Generalization in Generative Adversarial Networks: A Novel Perspective from Privacy Protection »
Bingzhe Wu · Shiwan Zhao · Chaochao Chen · Haoyang Xu · Li Wang · Xiaolu Zhang · Guangyu Sun · Jun Zhou