Timezone: »
It is commonplace to encounter heterogeneous data, of which some aspects of the data distribution may vary but the underlying causal mechanisms remain constant. When data are divided into distinct environments according to the heterogeneity, recent invariant learning methods have proposed to learn robust and invariant models using this environment partition. It is hence tempting to utilize the inherent heterogeneity even when environment partition is not provided. Unfortunately, in this work, we show that learning invariant features under this circumstance is fundamentally impossible without further inductive biases or additional information. Then, we propose a framework to jointly learn environment partition and invariant representation, assisted by additional auxiliary information. We derive sufficient and necessary conditions for our framework to provably identify invariant features under a fairly general setting. Experimental results on both synthetic and real world datasets validate our analysis and demonstrate an improved performance of the proposed framework. Our findings also raise the need of making the role of inductive biases more explicit when learning invariant models without environment partition in future works. Codes are available at https://github.com/linyongver/ZIN_official .
Author Information
Yong Lin (The Hong Kong University of Science and Technology)
I am an CSE PhD sudent in HKUST, supervised by Professor Tong Zhang. Out of Distribution Generalization , Robustess of Deep Learning and Learning theory are my research interests. Particularly, we are now working on topics related to Invariant Learning. If you are also interested in these fields or just my works, you can come to have a chat me.
Shengyu Zhu (Ubiquant)
Lu Tan (Tsinghua University)
Peng Cui (Tsinghua University)
Related Events (a corresponding poster, oral, or spotlight)
-
2022 Poster: ZIN: When and How to Learn Invariance Without Environment Partition? »
Dates n/a. Room
More from the Same Authors
-
2022 Poster: Para-CFlows: $C^k$-universal diffeomorphism approximators as superior neural surrogates »
Junlong Lyu · Zhitang Chen · Chang Feng · Wenjing Cun · Shengyu Zhu · Yanhui Geng · ZHIJIE XU · Chen Yongwei -
2022 Poster: Product Ranking for Revenue Maximization with Multiple Purchases »
Renzhe Xu · Xingxuan Zhang · Bo Li · Yafeng Zhang · Xiaolong Chen · Peng Cui -
2022 : Particle-based Variational Inference with Preconditioned Functional Gradient Flow »
Hanze Dong · Xi Wang · Yong Lin · Tong Zhang -
2023 Poster: ID and OOD Performance Are Sometimes Inversely Correlated on Real-world Datasets »
Damien Teney · Yong Lin · Seong Joon Oh · Ehsan Abbasnejad -
2023 Poster: On the Need for a Language Describing Distribution Shifts: Illustrations on Tabular Datasets »
Jiashuo Liu · Tianyu Wang · Peng Cui · Hongseok Namkoong -
2023 Tutorial: Exploring and Exploiting Data Heterogeneity for Prediction and Decision-Making »
Peng Cui · Jiashuo Liu · Hongseok Namkoong · Tianhui Cai -
2022 Spotlight: Lightning Talks 5B-4 »
Yuezhi Yang · Zeyu Yang · Yong Lin · Yishi Xu · Linan Yue · Tao Yang · Weixin Chen · Qi Liu · Jiaqi Chen · Dongsheng Wang · Baoyuan Wu · Yuwang Wang · Hao Pan · Shengyu Zhu · Zhenwei Miao · Yan Lu · Lu Tan · Bo Chen · Yichao Du · Haoqian Wang · Wei Li · Yanqing An · Ruiying Lu · Peng Cui · Nanning Zheng · Li Wang · Zhibin Duan · Xiatian Zhu · Mingyuan Zhou · Enhong Chen · Li Zhang -
2022 Spotlight: Lightning Talks 3A-2 »
shuwen yang · Xu Zhang · Delvin Ce Zhang · Lan-Zhe Guo · Renzhe Xu · Zhuoer Xu · Yao-Xiang Ding · Weihan Li · Xingxuan Zhang · Xi-Zhu Wu · Zhenyuan Yuan · Hady Lauw · Yu Qi · Yi-Ge Zhang · Zhihao Yang · Guanghui Zhu · Dong Li · Changhua Meng · Kun Zhou · Gang Pan · Zhi-Fan Wu · Bo Li · Minghui Zhu · Zhi-Hua Zhou · Yafeng Zhang · Yingxueff Zhang · shiwen cui · Jie-Jing Shao · Zhanguang Zhang · Zhenzhe Ying · Xiaolong Chen · Yu-Feng Li · Guojie Song · Peng Cui · Weiqiang Wang · Ming GU · Jianye Hao · Yihua Huang -
2022 Spotlight: Product Ranking for Revenue Maximization with Multiple Purchases »
Renzhe Xu · Xingxuan Zhang · Bo Li · Yafeng Zhang · Xiaolong Chen · Peng Cui -
2022 Spotlight: Lightning Talks 2B-3 »
Jie-Jing Shao · Jiangmeng Li · Jiashuo Liu · Zongbo Han · Tianyang Hu · Jiayun Wu · Wenwen Qiang · Jun WANG · Zhipeng Liang · Lan-Zhe Guo · Wenjia Wang · Yanan Zhang · Xiao-wen Yang · Fan Yang · Bo Li · Wenyi Mo · Zhenguo Li · Liu Liu · Peng Cui · Yu-Feng Li · Changwen Zheng · Lanqing Li · Yatao Bian · Bing Su · Hui Xiong · Peilin Zhao · Bingzhe Wu · Changqing Zhang · Jianhua Yao -
2022 Spotlight: Distributionally Robust Optimization with Data Geometry »
Jiashuo Liu · Jiayun Wu · Bo Li · Peng Cui -
2022 Poster: Distributionally Robust Optimization with Data Geometry »
Jiashuo Liu · Jiayun Wu · Bo Li · Peng Cui -
2021 Poster: Integrated Latent Heterogeneity and Invariance Learning in Kernel Space »
Jiashuo Liu · Zheyuan Hu · Peng Cui · Bo Li · Zheyan Shen -
2020 Poster: Counterfactual Prediction for Bundle Treatment »
Hao Zou · Peng Cui · Bo Li · Zheyan Shen · Jianxin Ma · Hongxia Yang · Yue He -
2019 Poster: Learning Disentangled Representations for Recommendation »
Jianxin Ma · Chang Zhou · Peng Cui · Hongxia Yang · Wenwu Zhu