Workshop
Mon Dec 13 09:00 AM -- 06:00 PM (PST)
Distribution shifts: connecting methods and applications (DistShift)
Distribution shifts---where a model is deployed on a data distribution different from what it was trained on---pose significant robustness challenges in real-world ML applications. Such shifts are often unavoidable in the wild and have been shown to substantially degrade model performance in applications such as biomedicine, wildlife conservation, sustainable development, robotics, education, and criminal justice. For example, models can systematically fail when tested on patients from different hospitals or people from different demographics. Despite the ubiquity of distribution shifts in ML applications, work on these types of real-world shifts is currently underrepresented in the ML research community, with prior work generally focusing instead on synthetic shifts. However, recent work has shown that models that are robust to one kind of shift need not be robust to another, underscoring the importance and urgency of studying the types of distribution shifts that arise in real-world ML deployments. With this workshop, we aim to facilitate deeper exchanges between domain experts in various ML application areas and more methods-oriented researchers, and ground the development of methods for characterizing and mitigating distribution shifts in real-world application contexts.
| Opening remarks (Talk) | |
| Distribution Shifts in AI for Social Good (Invited talk) | |
| Dataset Shifts: 8 Years of Going from Practice to Theory to Policy and Future Directions (Invited talk) | |
| ML Model Debugging: A Data Perspective (Invited talk) | |
| Discussion: Aleksander Mądry, Ernest Mwebaze, Suchi Saria (Panel) | |
| Increasing Robustness to Distribution Shifts by Improving Design (Invited talk) | |
| Statistical Testing under Distribution Shifts (Invited talk) | |
| Discussion: Elizabeth Tipton, Jonas Peters (Panel) | |
| Benchmarking Bayesian Deep Learning on Diabetic Retinopathy Detection Tasks (Spotlight) | |
| A Closer Look at Distribution Shifts and Out-of-Distribution Generalization on Graphs (Spotlight) | |
| On Adaptivity and Confounding in Contextual Bandit Experiments (Spotlight) | |
| Is Importance Weighting Incompatible with Interpolating Classifiers? (Spotlight) | |
| Poster session | |
| Break / Lounge (Break) | |
| Importance Weighting for Transfer Learning (Invited talk) | |
| Robustness through the Lens of Invariance (Invited talk) | |
| Discussion: Chelsea Finn, Masashi Sugiyama (Panel) | |
| Panel: Future directions for tackling distribution shifts (Panel) | |
| Shift and Scale is Detrimental To Few-Shot Transfer (Poster) | |
| Calibrated Ensembles: A Simple Way to Mitigate ID-OOD Accuracy Tradeoffs (Poster) | |
| Correct-N-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlations (Poster) | |
| Domain-agnostic Test-time Adaptation by Prototypical Training with Auxiliary Data (Poster) | |
| Select, Label, and Mix: Learning Discriminative Invariant Feature Representations for Partial Domain Adaptation (Poster) | |
| Spectrally Adaptive Common Spatial Patterns (Poster) | |
| How Does Contrastive Pre-training Connect Disparate Domains? (Poster) | |
| Handling Distribution Shift in Tire Design (Poster) | |
| Learning Invariant Representations with Missing Data (Poster) | |
| Multi-Domain Ensembles for Domain Generalization (Poster) | |
| Benchmarking Robustness to Natural Distribution Shifts for Facial Analysis (Poster) | |
| Distribution Shift in Airline Customer Behavior during COVID-19 (Poster) | |
| Extending the WILDS Benchmark for Unsupervised Adaptation (Poster) | |
| Tackling Online One-Class Incremental Learning by Removing Negative Contrasts (Poster) | |
| Avoiding Spurious Correlations: Bridging Theory and Practice (Poster) | |
| Model Zoo: A Growing Brain That Learns Continually (Poster) | |
| The impact of domain shift on the calibration of fine-tuned models (Poster) | |
| Probing Representation Forgetting in Continual Learning (Poster) | |
| Kernel Landmarks: An Empirical Statistical Approach to Detect Covariate Shift (Poster) | |
| A Unified DRO View of Multi-class Loss Functions with top-N Consistency (Poster) | |
| Re-labeling Domains Improves Multi-Domain Generalization (Poster) | |
| Using Distributionally Robust Optimization to improve robustness in cancer pathology (Poster) | |
| Thinking Beyond Distributions in Testing Machine Learned Models (Poster) | |
| Exploiting Causal Chains for Domain Generalization (Poster) | |
| KitchenShift: Evaluating Zero-Shot Generalization of Imitation-Based Policy Learning Under Domain Shifts (Poster) | |
| Distributionally Robust Group Backwards Compatibility (Poster) | |
| Understanding and Improving Robustness of VisionTransformers through patch-based NegativeAugmentation (Poster) | |
| Unsupervised Attribute Alignment for Characterizing Distribution Shift (Poster) | |
| Identifying the Instances Associated with Distribution Shifts using the Max-Sliced Bures Divergence (Poster) | |
| BEDS-Bench: Behavior of EHR-models under Distributional Shift - A Benchmark (Poster) | |
| Test Time Robustification of Deep Models via Adaptation and Augmentation (Poster) | |
| Catastrophic Failures of Neural Active Learning on Heteroskedastic Distributions (Poster) | |
| Jointly Learning from Decentralized (Federated) and Centralized Data to Mitigate Distribution Shift (Poster) | |
| Reliable Graph Neural Networks for Drug Discovery Under Distributional Shift (Poster) | |
| Distribution Preserving Bayesian Coresets using Set Constraints (Poster) | |
| Causal-based Time Series Domain Generalization for Vehicle Intention Prediction (Poster) | |
| Robust fine-tuning of zero-shot models (Poster) | |
| Are Vision Transformers Always More Robust Than Convolutional Neural Networks? (Poster) | |
| Newer is not always better: Rethinking transferability metrics, their peculiarities, stability and performance (Poster) | |
| Mix-MaxEnt: Improving Accuracy and Uncertainty Estimates of Deterministic Neural Networks (Poster) | |
| On The Reliability Of Machine Learning Applications In Manufacturing Environments (Poster) | |
| Igeood: An Information Geometry Approach to Out-of-Distribution Detection (Poster) | |
| Towards Data-Free Domain Generalization (Poster) | |
| Optimal Representations for Covariate Shifts (Poster) | |
| Continual Density Ratio Estimation (Poster) | |
| Gradient-matching coresets for continual learning (Poster) | |
| Distribution Mismatch Correction for Improved Robustness in Deep Neural Networks (Poster) | |
| Augmented Self-Labeling for Source-Free Unsupervised Domain Adaptation (Poster) | |
| An Empirical Study of Pre-trained Models on Out-of-distribution Generalization (Poster) | |
| Randomly projecting out distribution shifts for improved robustness (Poster) | |
| Semi-Supervised Domain Generalization with Stochastic StyleMatch (Poster) | |
| Understanding Post-hoc Adaptation for Improving Subgroup Robustness (Poster) | |
| Fourier-Based Augmentations for Improved Robustness and Uncertainty Calibration (Poster) | |
| Test time Adaptation through Perturbation Robustness (Poster) | |
| Diurnal or Nocturnal? Federated Learning from Periodically Shifting Distributions (Poster) | |
| Maximum Mean Discrepancy for Generalization in the Presence of Distribution and Missingness Shift (Poster) | |
| A fine-grained analysis of robustness to distribution shifts (Poster) | |
| Is Importance Weighting Incompatible with Interpolating Classifiers? (Poster) | |
| Surprisingly Simple Semi-Supervised Domain Adaptation with Pretraining and Consistency (Poster) | |
| Self-supervised Learning is More Robust to Dataset Imbalance (Poster) | |
| Effect of Model Size on Worst-group Generalization (Poster) | |
| Smooth Transfer Learning for Source-to-Target Generalization (Poster) | |
| An Empirical Investigation of Model-to-Model Distribution Shifts in Trained Convolutional Filters (Poster) | |
| DANNTe: a case study of a turbo-machinery sensor virtualization under domain shift (Poster) | |
| Internalized Biases in Fréchet Inception Distance (Poster) | |
| Investigating Shifts in GAN Output-Distributions (Poster) | |
| Con$^{2}$DA: Simplifying Semi-supervised Domain Adaptation by Learning Consistent and Contrastive Feature Representations (Poster) | |
| Benchmarking Bayesian Deep Learning on Diabetic Retinopathy Detection Tasks (Poster) | |
| On Adaptivity and Confounding in Contextual Bandit Experiments (Poster) | |
| Exploring Covariate and Concept Shift for Out-of-Distribution Detection (Poster) | |
| Just Mix Once: Mixing Samples with Implicit Group Distribution (Poster) | |
| PCA Subspaces Are Not Always Optimal for Bayesian Learning (Poster) | |
| PixMix: Dreamlike Pictures Comprehensively Improve Safety Measures (Poster) | |
| Quantifying and Alleviating Distribution Shifts in Foundation Models on Review Classification (Poster) | |
| Mixture of Basis for Interpretable Continual Learning with Distribution Shifts (Poster) | |
| Ensembles and Cocktails: Robust Finetuning for Natural Language Generation (Poster) | |
| A benchmark with decomposed distribution shifts for 360 monocular depth estimation (Poster) | |
| Leveraging Unlabeled Data to Predict Out-of-Distribution Performance (Poster) | |
| Improving Baselines in the Wild (Poster) | |
| Boosting worst-group accuracy without group annotations (Poster) | |
| Adversarial Training Blocks Generalization in Neural Policies (Poster) | |
| Revisiting Visual Product for Compositional Zero-Shot Learning (Poster) | |
| Nonparametric Approach to Uncertainty Quantification for Deterministic Neural Networks (Poster) | |
| Towards Robust and Adaptable Motion Forecasting: A Causal Representation Perspective (Poster) | |
| A Closer Look at Distribution Shifts and Out-of-Distribution Generalization on Graphs (Poster) | |