Unraveling the Effects of Age-Based Distribution Shifts on Medical Image Classifiers

Medical AI underperforms when faced with new data distributions. A notorious example of this is the Epic Sepsis Model (ESM), whose predictive capabilities diminished upon testing with unseen patient data distributions. We study the effects of subpopulation shifts on medical image classifiers using two representative datasets. Our results highlight the nuanced effects of class distribution imbalances on performance drops, the significance of comprehensive evaluation strategies, and the need to collect diverse samples to advance medical AI deployment.