Timezone: »
Self-supervised representation learning has shown remarkable success in a number of domains. A common practice is to perform data augmentation via hand-crafted transformations intended to leave the semantics of the data invariant. We seek to understand the empirical success of this approach from a theoretical perspective. We formulate the augmentation process as a latent variable model by postulating a partition of the latent representation into a content component, which is assumed invariant to augmentation, and a style component, which is allowed to change. Unlike prior work on disentanglement and independent component analysis, we allow for both nontrivial statistical and causal dependencies in the latent space. We study the identifiability of the latent representation based on pairs of views of the observations and prove sufficient conditions that allow us to identify the invariant content partition up to an invertible mapping in both generative and discriminative settings. We find numerical simulations with dependent latent variables are consistent with our theory. Lastly, we introduce Causal3DIdent, a dataset of high-dimensional, visually complex images with rich causal dependencies, which we use to study the effect of data augmentations performed in practice.
Author Information
Julius von Kügelgen (Max Planck Institute for Intelligent Systems Tübingen & University of Cambridge)
Yash Sharma (University of Tübingen)
Luigi Gresele (MPI for Intelligent Systems, Tübingen)
Wieland Brendel (AG Bethge, University of Tübingen)
Bernhard Schölkopf (MPI for Intelligent Systems, Tübingen)
Michel Besserve (MPI for Intelligent Systems, Tübingen)
Francesco Locatello (Amazon)
More from the Same Authors
-
2021 Spotlight: Iterative Teaching by Label Synthesis »
Weiyang Liu · Zhen Liu · Hanchen Wang · Liam Paull · Bernhard Schölkopf · Adrian Weller -
2021 Spotlight: How Well do Feature Visualizations Support Causal Understanding of CNN Activations? »
Roland S. Zimmermann · Judy Borowski · Robert Geirhos · Matthias Bethge · Thomas Wallis · Wieland Brendel -
2021 Spotlight: DiBS: Differentiable Bayesian Structure Learning »
Lars Lorch · Jonas Rothfuss · Bernhard Schölkopf · Andreas Krause -
2022 : Active Bayesian Causal Inference »
Christian Toth · Lars Lorch · Christian Knoll · Andreas Krause · Franz Pernkopf · Robert Peharz · Julius von Kügelgen -
2022 : A Causal Framework to Quantify Robustness of Mathematical Reasoning with Language Models »
Alessandro Stolfo · Zhijing Jin · Kumar Shridhar · Bernhard Schölkopf · Mrinmaya Sachan -
2022 : Evaluating vaccine allocation strategies using simulation-assisted causal modelling »
Armin Kekić · Jonas Dehning · Luigi Gresele · Julius von Kügelgen · Viola Priesemann · Bernhard Schölkopf -
2022 : Active Bayesian Causal inference »
Christian Toth · Lars Lorch · Christian Knoll · Andreas Krause · Franz Pernkopf · Robert Peharz · Julius von Kügelgen -
2022 : A General-Purpose Neural Architecture for Geospatial Systems »
Martin Weiss · Nasim Rahaman · Frederik Träuble · Francesco Locatello · Alexandre Lacoste · Yoshua Bengio · Erran Li Li · Chris Pal · Bernhard Schölkopf -
2023 Poster: Error Bounds for Score Matching Causal Discovery »
Zhenyu Zhu · Francesco Locatello · Volkan Cevher -
2023 Poster: Latent Space Translation via Semantic Alignment »
Valentino Maiorca · Luca Moschella · Antonio Norelli · Marco Fumero · Francesco Locatello · Emanuele Rodolà -
2023 Poster: Rotating Features for Object Discovery »
Sindy Löwe · Phillip Lippe · Francesco Locatello · Max Welling -
2023 Poster: CLadder: Assessing Causal Reasoning in Language Models »
Zhijing Jin · Yuen Chen · Felix Leeb · Luigi Gresele · Ojasv Kamal · Zhiheng LYU · Kevin Blin · Fernando Gonzalez Adauto · Max Kleiman-Weiner · Mrinmaya Sachan · Bernhard Schölkopf -
2023 Poster: ASIF: Coupled Data Turns Unimodal Models to Multimodal without Training »
Antonio Norelli · Marco Fumero · Valentino Maiorca · Luca Moschella · Emanuele Rodolà · Francesco Locatello -
2023 Poster: Nonparametric Identifiability of Causal Representations from Unknown Interventions »
Julius von Kügelgen · Michel Besserve · Liang Wendong · Luigi Gresele · Armin Kekić · Elias Bareinboim · David Blei · Bernhard Schölkopf -
2023 Poster: Assumption violations in causal discovery and the robustness of score matching »
Francesco Montagna · Atalanti Mastakouri · Elias Eulig · Nicoletta Noceti · Lorenzo Rosasco · Dominik Janzing · Bryon Aragam · Francesco Locatello -
2023 Poster: Leveraging sparse and shared feature activations for disentangled representation learning »
Marco Fumero · Florian Wenzel · Luca Zancato · Alessandro Achille · Emanuele Rodolà · Stefano Soatto · Bernhard Schölkopf · Francesco Locatello -
2023 Poster: Causal Component Analysis »
Liang Wendong · Armin Kekić · Julius von Kügelgen · Simon Buchholz · Michel Besserve · Luigi Gresele · Bernhard Schölkopf -
2023 Poster: On Transfer of Adversarial Robustness from Pretraining to Downstream Tasks »
Laura F. Nern · Harsh Raj · Maurice Georgi · Yash Sharma -
2023 Poster: Spuriosity Didn’t Kill the Classifier: Using Invariant Predictions to Harness Spurious Features »
Cian Eastwood · Shashank Singh · Andrei L Nicolicioiu · Marin Vlastelica Pogančić · Julius von Kügelgen · Bernhard Schölkopf -
2023 Oral: Rotating Features for Object Discovery »
Sindy Löwe · Phillip Lippe · Francesco Locatello · Max Welling -
2023 Workshop: UniReps: Unifying Representations in Neural Models »
Marco Fumero · Emanuele Rodolà · Francesco Locatello · Gintare Karolina Dziugaite · Mathilde Caron · Clémentine Dominé -
2022 Spotlight: Lightning Talks 1A-3 »
Kimia Noorbakhsh · Ronan Perry · Qi Lyu · Jiawei Jiang · Christian Toth · Olivier Jeunen · Xin Liu · Yuan Cheng · Lei Li · Manuel Rodriguez · Julius von Kügelgen · Lars Lorch · Nicolas Donati · Lukas Burkhalter · Xiao Fu · Zhongdao Wang · Songtao Feng · Ciarán Gilligan-Lee · Rishabh Mehrotra · Fangcheng Fu · Jing Yang · Bernhard Schölkopf · Ya-Li Li · Christian Knoll · Maks Ovsjanikov · Andreas Krause · Shengjin Wang · Hong Zhang · Mounia Lalmas · Bolin Ding · Bo Du · Yingbin Liang · Franz Pernkopf · Robert Peharz · Anwar Hithnawi · Julius von Kügelgen · Bo Li · Ce Zhang -
2022 Spotlight: Active Bayesian Causal Inference »
Christian Toth · Lars Lorch · Christian Knoll · Andreas Krause · Franz Pernkopf · Robert Peharz · Julius von Kügelgen -
2022 Spotlight: Causal Discovery in Heterogeneous Environments Under the Sparse Mechanism Shift Hypothesis »
Ronan Perry · Julius von Kügelgen · Bernhard Schölkopf -
2022 Spotlight: Embrace the Gap: VAEs Perform Independent Mechanism Analysis »
Patrik Reizinger · Luigi Gresele · Jack Brady · Julius von Kügelgen · Dominik Zietlow · Bernhard Schölkopf · Georg Martius · Wieland Brendel · Michel Besserve -
2022 Poster: Probable Domain Generalization via Quantile Risk Minimization »
Cian Eastwood · Alexander Robey · Shashank Singh · Julius von Kügelgen · Hamed Hassani · George J. Pappas · Bernhard Schölkopf -
2022 Poster: Causal Discovery in Heterogeneous Environments Under the Sparse Mechanism Shift Hypothesis »
Ronan Perry · Julius von Kügelgen · Bernhard Schölkopf -
2022 Poster: Increasing Confidence in Adversarial Robustness Evaluations »
Roland S. Zimmermann · Wieland Brendel · Florian Tramer · Nicholas Carlini -
2022 Poster: Embrace the Gap: VAEs Perform Independent Mechanism Analysis »
Patrik Reizinger · Luigi Gresele · Jack Brady · Julius von Kügelgen · Dominik Zietlow · Bernhard Schölkopf · Georg Martius · Wieland Brendel · Michel Besserve -
2022 Poster: Active Bayesian Causal Inference »
Christian Toth · Lars Lorch · Christian Knoll · Andreas Krause · Franz Pernkopf · Robert Peharz · Julius von Kügelgen -
2021 : Boxhead: A Dataset for Learning Hierarchical Representations »
Yukun Chen · Andrea Dittadi · Frederik Träuble · Stefan Bauer · Bernhard Schölkopf -
2021 : Julius von Kügelgen - Independent mechanism analysis, a new concept? »
Julius von Kügelgen -
2021 Poster: How Well do Feature Visualizations Support Causal Understanding of CNN Activations? »
Roland S. Zimmermann · Judy Borowski · Robert Geirhos · Matthias Bethge · Thomas Wallis · Wieland Brendel -
2021 Poster: Dynamic Inference with Neural Interpreters »
Nasim Rahaman · Muhammad Waleed Gondal · Shruti Joshi · Peter Gehler · Yoshua Bengio · Francesco Locatello · Bernhard Schölkopf -
2021 Poster: Unsupervised Learning of Compositional Energy Concepts »
Yilun Du · Shuang Li · Yash Sharma · Josh Tenenbaum · Igor Mordatch -
2021 Poster: Causal Influence Detection for Improving Efficiency in Reinforcement Learning »
Maximilian Seitzer · Bernhard Schölkopf · Georg Martius -
2021 Poster: Independent mechanism analysis, a new concept? »
Luigi Gresele · Julius von Kügelgen · Vincent Stimper · Bernhard Schölkopf · Michel Besserve -
2021 Oral: Partial success in closing the gap between human and machine vision »
Robert Geirhos · Kantharaju Narayanappa · Benjamin Mitzkus · Tizian Thieringer · Matthias Bethge · Felix A. Wichmann · Wieland Brendel -
2021 Poster: Iterative Teaching by Label Synthesis »
Weiyang Liu · Zhen Liu · Hanchen Wang · Liam Paull · Bernhard Schölkopf · Adrian Weller -
2021 Poster: Partial success in closing the gap between human and machine vision »
Robert Geirhos · Kantharaju Narayanappa · Benjamin Mitzkus · Tizian Thieringer · Matthias Bethge · Felix A. Wichmann · Wieland Brendel -
2021 Poster: The Inductive Bias of Quantum Kernels »
Jonas Kübler · Simon Buchholz · Bernhard Schölkopf -
2021 Poster: Backward-Compatible Prediction Updates: A Probabilistic Approach »
Frederik Träuble · Julius von Kügelgen · Matthäus Kleindessner · Francesco Locatello · Bernhard Schölkopf · Peter Gehler -
2021 Poster: Fast Minimum-norm Adversarial Attacks through Adaptive Norm Constraints »
Maura Pintor · Fabio Roli · Wieland Brendel · Battista Biggio -
2021 Poster: DiBS: Differentiable Bayesian Structure Learning »
Lars Lorch · Jonas Rothfuss · Bernhard Schölkopf · Andreas Krause -
2021 Poster: Regret Bounds for Gaussian-Process Optimization in Large Domains »
Manuel Wuethrich · Bernhard Schölkopf · Andreas Krause -
2020 Poster: Modeling Shared responses in Neuroimaging Studies through MultiView ICA »
Hugo Richard · Luigi Gresele · Aapo Hyvarinen · Bertrand Thirion · Alexandre Gramfort · Pierre Ablin -
2020 Spotlight: Modeling Shared responses in Neuroimaging Studies through MultiView ICA »
Hugo Richard · Luigi Gresele · Aapo Hyvarinen · Bertrand Thirion · Alexandre Gramfort · Pierre Ablin -
2020 Poster: Relative gradient optimization of the Jacobian term in unsupervised deep learning »
Luigi Gresele · Giancarlo Fissore · Adrián Javaloy · Bernhard Schölkopf · Aapo Hyvarinen -
2020 Poster: Improving robustness against common corruptions by covariate shift adaptation »
Steffen Schneider · Evgenia Rusak · Luisa Eck · Oliver Bringmann · Wieland Brendel · Matthias Bethge -
2019 : Bernhard Schölkopf »
Bernhard Schölkopf -
2019 Poster: Learning from brains how to regularize machines »
Zhe Li · Wieland Brendel · Edgar Walker · Erick Cobos · Taliah Muhammad · Jacob Reimer · Matthias Bethge · Fabian Sinz · Xaq Pitkow · Andreas Tolias -
2019 Poster: Accurate, reliable and fast robustness evaluation »
Wieland Brendel · Jonas Rauber · Matthias Kümmerer · Ivan Ustyuzhaninov · Matthias Bethge -
2018 : Adversarial Vision Challenge: Results of the Adversarial Vision Challenge »
Wieland Brendel · Jonas Rauber · Marcel Salathé · Alexey Kurakin · Nicolas Papernot · Sharada Mohanty · Matthias Bethge -
2018 : Learning Independent Mechanisms »
Bernhard Schölkopf