How Much Distribution Shift Changes a Conclusion?
Jonathan Zhou ⋅ Rohan Kumar
Abstract
In applied statistics, identified model parameters often inform scientific, economic, business, and policy decisions. Yet conclusions drawn from study data may be sensitive to sampling variability, contamination, measurement error, or other forms of distribution shift. We study this sensitivity through a Distributionally Robust Optimization (DRO) lens by defining a distributionally robust conclusion interval: the range of conclusions attainable under structured perturbations of the observed distribution within a prescribed transportation budget. The framework accommodates design-specific restrictions on admissible perturbations. For conclusions induced by $Z$-estimators—a broad class encompassing many regression and moment-based estimators—we approximately compute the interval endpoints using a cutting-plane algorithm whose separation problem decomposes across observations and can be solved in parallel. Applications in healthcare, economics, and GenAI illustrate how the resulting diagnostic can reveal both conclusion fragility and the perturbations responsible for it.
Chat is not available.
Successful Page Load