Timezone: »
We develop methods for forming prediction sets in an online setting where the data generating distribution is allowed to vary over time in an unknown fashion. Our framework builds on ideas from conformal inference to provide a general wrapper that can be combined with any black box method that produces point predictions of the unseen label or estimated quantiles of its distribution. While previous conformal inference methods rely on the assumption that the data are exchangeable, our adaptive approach provably achieves the desired coverage frequency over long-time intervals irrespective of the true data generating process. We accomplish this by modelling the distribution shift as a learning problem in a single parameter whose optimal value is varying over time and must be continuously re-estimated. We test our method, adaptive conformal inference, on two real world datasets and find that its predictions are robust to visible and significant distribution shifts.
Author Information
Isaac Gibbs (Stanford University)
Emmanuel Candes (Stanford University)
Related Events (a corresponding poster, oral, or spotlight)
-
2021 Oral: Adaptive Conformal Inference Under Distribution Shift »
Sat. Dec 11th 12:40 -- 12:55 AM Room
More from the Same Authors
-
2023 Poster: Conformal Scorecasting: Anticipatory Uncertainty Quantification for Distribution Shift in Time Series »
Anastasios Angelopoulos · Ryan Tibshirani · Emmanuel Candes -
2023 Poster: Uncertainty Quantification over Graph with Conformalized Graph Neural Networks »
Kexin Huang · Ying Jin · Emmanuel Candes · Jure Leskovec -
2022 Invited Talk: Conformal Prediction in 2022 »
Emmanuel Candes -
2020 Poster: Achieving Equalized Odds by Resampling Sensitive Attributes »
Yaniv Romano · Stephen Bates · Emmanuel Candes -
2020 Poster: Classification with Valid and Adaptive Coverage »
Yaniv Romano · Matteo Sesia · Emmanuel Candes -
2020 Spotlight: Classification with Valid and Adaptive Coverage »
Yaniv Romano · Matteo Sesia · Emmanuel Candes -
2019 Poster: Conformalized Quantile Regression »
Yaniv Romano · Evan Patterson · Emmanuel Candes -
2019 Poster: Conformal Prediction Under Covariate Shift »
Ryan Tibshirani · Rina Barber · Emmanuel Candes · Aaditya Ramdas -
2015 Poster: Solving Random Quadratic Systems of Equations Is Nearly as Easy as Solving Linear Systems »
Yuxin Chen · Emmanuel Candes -
2015 Oral: Solving Random Quadratic Systems of Equations Is Nearly as Easy as Solving Linear Systems »
Yuxin Chen · Emmanuel Candes -
2014 Poster: A Differential Equation for Modeling Nesterov’s Accelerated Gradient Method: Theory and Insights »
Weijie Su · Stephen Boyd · Emmanuel Candes -
2014 Spotlight: A Differential Equation for Modeling Nesterov’s Accelerated Gradient Method: Theory and Insights »
Weijie Su · Stephen Boyd · Emmanuel Candes