Timezone: »
The deployment of machine learning models in safety-critical applications comes with the expectation that such models will perform well over a range of contexts (e.g., a vision model for classifying street signs should work in rural, city, and highway settings under varying lighting/weather conditions). However, these one-size-fits-all models are typically optimized for average case performance, encouraging them to achieve high performance in nominal conditions but exposing them to unexpected behavior in challenging or rare contexts. To address this concern, we develop a new method for training context-dependent models. We extend Bridge-Mode Connectivity (BMC) to train an infinite ensemble of models over a continuous measure of context such that we can sample model parameters specifically tuned to the corresponding evaluation context. We explore the definition of context in image classification tasks through multiple lenses including changes in the risk profile, long-tail image statistics/appearance, and context-dependent distribution shift. We develop novel extensions of the BMC optimization for each of these cases and our experiments demonstrate that model performance can be successfully tuned to context in each scenario.
Author Information
Nathan Drenkow (The Johns Hopkins University Applied Physics Laboratory)
Alvin Tan (UC Berkeley)
Clayton Ashcraft (Johns Hopkins University Applied Physics Lab)
Kiran Karra (JHU/APL)
More from the Same Authors
-
2022 : Fifteen-minute Competition Overview Video »
Nathan Drenkow · Raman Arora · Gino Perrotta · Todd Neller · Ryan Gardner · Mykel J Kochenderfer · Jared Markowitz · Corey Lowman · Casey Richardson · Bo Li · Bart Paulhamus · Ashley J Llorens · Andrew Newman -
2022 : Cortical Transformers: Robustness and Model Compression with Multi-Scale Connectivity Properties of the Neocortex. »
Brian Robinson · Nathan Drenkow -
2022 : Machine Learning for Activity-Based Road Transportation Emissions Estimation »
Derek Rollend · Kevin Foster · Tomek Kott · Rohita Mocharla · Rodrigo Rene Rai Muñoz Abujder · Neil Fendley · Clayton Ashcraft · Frank Willard · Marisa Hughes -
2022 Competition: Reconnaissance Blind Chess: An Unsolved Challenge for Multi-Agent Decision Making Under Uncertainty »
Ryan Gardner · Gino Perrotta · Corey Lowman · Casey Richardson · Andrew Newman · Jared Markowitz · Nathan Drenkow · Bart Paulhamus · Ashley J Llorens · Todd Neller · Raman Arora · Bo Li · Mykel J Kochenderfer -
2021 : Reconnaissance Blind Chess + Q&A »
Ryan Gardner · Gino Perrotta · Corey Lowman · Casey Richardson · Andrew Newman · Jared Markowitz · Nathan Drenkow · Bart Paulhamus · Ashley J Llorens · Todd Neller · Raman Arora · Bo Li · Mykel J Kochenderfer -
2017 : bossDB: A Petascale Database for Large-Scale Neuroscience Informing Machine Learning »
Nathan Drenkow