Workshop
Mon Dec 13 09:00 AM -- 06:00 PM (PST)
Distribution shifts: connecting methods and applications (DistShift)
Distribution shifts---where a model is deployed on a data distribution different from what it was trained on---pose significant robustness challenges in real-world ML applications. Such shifts are often unavoidable in the wild and have been shown to substantially degrade model performance in applications such as biomedicine, wildlife conservation, sustainable development, robotics, education, and criminal justice. For example, models can systematically fail when tested on patients from different hospitals or people from different demographics. Despite the ubiquity of distribution shifts in ML applications, work on these types of real-world shifts is currently underrepresented in the ML research community, with prior work generally focusing instead on synthetic shifts. However, recent work has shown that models that are robust to one kind of shift need not be robust to another, underscoring the importance and urgency of studying the types of distribution shifts that arise in real-world ML deployments. With this workshop, we aim to facilitate deeper exchanges between domain experts in various ML application areas and more methods-oriented researchers, and ground the development of methods for characterizing and mitigating distribution shifts in real-world application contexts.
| Opening remarks (Talk) | |
| Distribution Shifts in AI for Social Good (Invited talk) | |
| Dataset Shifts: 8 Years of Going from Practice to Theory to Policy and Future Directions (Invited talk) | |
| ML Model Debugging: A Data Perspective (Invited talk) | |
| Discussion: Aleksander Mądry, Ernest Mwebaze, Suchi Saria (Panel) | |
| Increasing Robustness to Distribution Shifts by Improving Design (Invited talk) | |
| Statistical Testing under Distribution Shifts (Invited talk) | |
| Discussion: Elizabeth Tipton, Jonas Peters (Panel) | |
| Benchmarking Bayesian Deep Learning on Diabetic Retinopathy Detection Tasks (Spotlight) | |
| A Closer Look at Distribution Shifts and Out-of-Distribution Generalization on Graphs (Spotlight) | |
| On Adaptivity and Confounding in Contextual Bandit Experiments (Spotlight) | |
| Is Importance Weighting Incompatible with Interpolating Classifiers? (Spotlight) | |
| Poster session | |
| Break / Lounge (Break) | |
| Importance Weighting for Transfer Learning (Invited talk) | |
| Robustness through the Lens of Invariance (Invited talk) | |
| Discussion: Chelsea Finn, Masashi Sugiyama (Panel) | |
| Panel: Future directions for tackling distribution shifts (Panel) | |
| Identifying the Instances Associated with Distribution Shifts using the Max-Sliced Bures Divergence (Poster) | |
| Distribution Preserving Bayesian Coresets using Set Constraints (Poster) | |
| Causal-based Time Series Domain Generalization for Vehicle Intention Prediction (Poster) | |
| Are Vision Transformers Always More Robust Than Convolutional Neural Networks? (Poster) | |
| Newer is not always better: Rethinking transferability metrics, their peculiarities, stability and performance (Poster) | |
| Mix-MaxEnt: Improving Accuracy and Uncertainty Estimates of Deterministic Neural Networks (Poster) | |
| Igeood: An Information Geometry Approach to Out-of-Distribution Detection (Poster) | |
| Towards Data-Free Domain Generalization (Poster) | |
| Augmented Self-Labeling for Source-Free Unsupervised Domain Adaptation (Poster) | |
| An Empirical Study of Pre-trained Models on Out-of-distribution Generalization (Poster) | |
| Understanding Post-hoc Adaptation for Improving Subgroup Robustness (Poster) | |
| Optimal Representations for Covariate Shifts (Poster) | |
| Maximum Mean Discrepancy for Generalization in the Presence of Distribution and Missingness Shift (Poster) | |
| A fine-grained analysis of robustness to distribution shifts (Poster) | |
| Surprisingly Simple Semi-Supervised Domain Adaptation with Pretraining and Consistency (Poster) | |
| Smooth Transfer Learning for Source-to-Target Generalization (Poster) | |
| Con$^{2}$DA: Simplifying Semi-supervised Domain Adaptation by Learning Consistent and Contrastive Feature Representations (Poster) | |
| Select, Label, and Mix: Learning Discriminative Invariant Feature Representations for Partial Domain Adaptation (Poster) | |
| Benchmarking Bayesian Deep Learning on Diabetic Retinopathy Detection Tasks (Poster) | |
| Distribution Mismatch Correction for Improved Robustness in Deep Neural Networks (Poster) | |
| Boosting worst-group accuracy without group annotations (Poster) | |
| Effect of Model Size on Worst-group Generalization (Poster) | |
| Model Zoo: A Growing Brain That Learns Continually (Poster) | |
| Catastrophic Failures of Neural Active Learning on Heteroskedastic Distributions (Poster) | |
| Thinking Beyond Distributions in Testing Machine Learned Models (Poster) | |
| Reliable Graph Neural Networks for Drug Discovery Under Distributional Shift (Poster) | |
| PixMix: Dreamlike Pictures Comprehensively Improve Safety Measures (Poster) | |
| Ensembles and Cocktails: Robust Finetuning for Natural Language Generation (Poster) | |
| Leveraging Unlabeled Data to Predict Out-of-Distribution Performance (Poster) | |
| Improving Baselines in the Wild (Poster) | |
| Nonparametric Approach to Uncertainty Quantification for Deterministic Neural Networks (Poster) | |
| Spectrally Adaptive Common Spatial Patterns (Poster) | |
| Multi-Domain Ensembles for Domain Generalization (Poster) | |
| Tackling Online One-Class Incremental Learning by Removing Negative Contrasts (Poster) | |
| Avoiding Spurious Correlations: Bridging Theory and Practice (Poster) | |
| The impact of domain shift on the calibration of fine-tuned models (Poster) | |
| Probing Representation Forgetting in Continual Learning (Poster) | |
| Using Distributionally Robust Optimization to improve robustness in cancer pathology (Poster) | |
| Understanding and Improving Robustness of VisionTransformers through patch-based NegativeAugmentation (Poster) | |
| Unsupervised Attribute Alignment for Characterizing Distribution Shift (Poster) | |
| BEDS-Bench: Behavior of EHR-models under Distributional Shift - A Benchmark (Poster) | |
| Test Time Robustification of Deep Models via Adaptation and Augmentation (Poster) | |
| Jointly Learning from Decentralized (Federated) and Centralized Data to Mitigate Distribution Shift (Poster) | |
| On The Reliability Of Machine Learning Applications In Manufacturing Environments (Poster) | |
| Semi-Supervised Domain Generalization with Stochastic StyleMatch (Poster) | |
| Test time Adaptation through Perturbation Robustness (Poster) | |
| Diurnal or Nocturnal? Federated Learning from Periodically Shifting Distributions (Poster) | |
| Is Importance Weighting Incompatible with Interpolating Classifiers? (Poster) | |
| Investigating Shifts in GAN Output-Distributions (Poster) | |
| On Adaptivity and Confounding in Contextual Bandit Experiments (Poster) | |
| Revisiting Visual Product for Compositional Zero-Shot Learning (Poster) | |
| Towards Robust and Adaptable Motion Forecasting: A Causal Representation Perspective (Poster) | |
| Shift and Scale is Detrimental To Few-Shot Transfer (Poster) | |
| Correct-N-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlations (Poster) | |
| Learning Invariant Representations with Missing Data (Poster) | |
| Extending the WILDS Benchmark for Unsupervised Adaptation (Poster) | |
| A Unified DRO View of Multi-class Loss Functions with top-N Consistency (Poster) | |
| Exploring Covariate and Concept Shift for Out-of-Distribution Detection (Poster) | |
| Robust fine-tuning of zero-shot models (Poster) | |
| Continual Density Ratio Estimation (Poster) | |
| Gradient-matching coresets for continual learning (Poster) | |
| Randomly projecting out distribution shifts for improved robustness (Poster) | |
| Fourier-Based Augmentations for Improved Robustness and Uncertainty Calibration (Poster) | |
| Self-supervised Learning is More Robust to Dataset Imbalance (Poster) | |
| An Empirical Investigation of Model-to-Model Distribution Shifts in Trained Convolutional Filters (Poster) | |
| DANNTe: a case study of a turbo-machinery sensor virtualization under domain shift (Poster) | |
| Internalized Biases in Fréchet Inception Distance (Poster) | |
| KitchenShift: Evaluating Zero-Shot Generalization of Imitation-Based Policy Learning Under Domain Shifts (Poster) | |
| Just Mix Once: Mixing Samples with Implicit Group Distribution (Poster) | |
| PCA Subspaces Are Not Always Optimal for Bayesian Learning (Poster) | |
| Quantifying and Alleviating Distribution Shifts in Foundation Models on Review Classification (Poster) | |
| Mixture of Basis for Interpretable Continual Learning with Distribution Shifts (Poster) | |
| A benchmark with decomposed distribution shifts for 360 monocular depth estimation (Poster) | |
| Adversarial Training Blocks Generalization in Neural Policies (Poster) | |
| A Closer Look at Distribution Shifts and Out-of-Distribution Generalization on Graphs (Poster) | |
| Calibrated Ensembles: A Simple Way to Mitigate ID-OOD Accuracy Tradeoffs (Poster) | |
| Domain-agnostic Test-time Adaptation by Prototypical Training with Auxiliary Data (Poster) | |
| How Does Contrastive Pre-training Connect Disparate Domains? (Poster) | |
| Handling Distribution Shift in Tire Design (Poster) | |
| Benchmarking Robustness to Natural Distribution Shifts for Facial Analysis (Poster) | |
| Distribution Shift in Airline Customer Behavior during COVID-19 (Poster) | |
| Kernel Landmarks: An Empirical Statistical Approach to Detect Covariate Shift (Poster) | |
| Re-labeling Domains Improves Multi-Domain Generalization (Poster) | |
| Exploiting Causal Chains for Domain Generalization (Poster) | |
| Distributionally Robust Group Backwards Compatibility (Poster) | |