Timezone: »
Large language models often err during deployment due to non-representative training data or distribution shift in the test set. Recently, model editors have been proposed to fix errors by adjusting a pre-trained model's weights. However, these approaches quickly decay a model's performance on upstream data, and forget how to fix previous errors. We propose and study a novel Lifelong Model Editing setting, where errors stream into a deployed model and we update the model to correct its predictions without influencing it for unrelated inputs. We propose General Retrieval Adaptors for Continual Editing, or GRACE, which learns and caches a particular layer's activations in a codebook as edits stream in, while the original model weights remain frozen. This ensures similar edits are treated similarly without altering the model's performance on unrelated instances. Experimentally, we show that GRACE substantially improves over recent model editors.
Author Information
Thomas Hartvigsen (Massachusetts Institute of Technology)
Swami Sankaranarayanan (Massachusetts Institute of Technology)
Hamid Palangi (Microsoft Research)
Yoon Kim (Massachusetts Institute of Technology)
Marzyeh Ghassemi (MIT)
Related Events (a corresponding poster, oral, or spotlight)
-
2022 : Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors »
Fri. Dec 2nd 08:25 -- 08:35 PM Room
More from the Same Authors
-
2021 : Improving the Fairness of Deep Chest X-ray Classifiers »
Haoran Zhang · Natalie Dullerud · Karsten Roth · Stephen Pfohl · Marzyeh Ghassemi -
2022 : Multimodal Checklists for Fair Clinical Decision Support »
Qixuan Jin · Marzyeh Ghassemi -
2022 : Deep Metric Learning to predict cardiac pressure with ECG »
Hyewon Jeong · Marzyeh Ghassemi · Collin Stultz -
2022 : Identifying Disparities in Sepsis Treatment using Inverse Reinforcement Learning »
Hyewon Jeong · Taylor Killian · Sanjat Kanjilal · Siddharth Nayak · Marzyeh Ghassemi -
2022 : Evaluating and Improving Robustness of Self-Supervised Representations to Spurious Correlations »
Kimia Hamidieh · Haoran Zhang · Marzyeh Ghassemi -
2022 : Learning to Defer in Ranking Systems »
Aparna Balagopalan · Haoran Zhang · Elizabeth Bondi-Kelly · Thomas Hartvigsen · Marzyeh Ghassemi -
2022 : Fair Active learning by exploiting causal data structure »
Sindhu C M Gowda · Haoran Zhang · Marzyeh Ghassemi -
2022 : Evaluation of Active Learning and Domain Adaptation on Health Data »
Kristina Holsapple · Haoran Zhang · Marzyeh Ghassemi -
2022 : Feature Restricted Group Dropout for Robust Electronic Health Record Predictions »
Bret Nestor · Anna Goldenberg · Marzyeh Ghassemi -
2022 : Identifying Disparities in Sepsis Treatment by Learning the Expert Policy »
Hyewon Jeong · Siddharth Nayak · Taylor Killian · Sanjat Kanjilal · Marzyeh Ghassemi -
2022 : Identifying Disparities in Sepsis Treatment by Learning the Expert Policy »
Hyewon Jeong · Siddharth Nayak · Taylor Killian · Sanjat Kanjilal · Marzyeh Ghassemi -
2022 : PEST: Combining Parameter-Efficient Fine-Tuning with Self-Training and Co-Training »
Hunter Lang · Monica Agrawal · Yoon Kim · David Sontag -
2022 : "Why did the Model Fail?": Attributing Model Performance Changes to Distribution Shifts »
Haoran Zhang · Harvineet Singh · Marzyeh Ghassemi · Shalmali Joshi -
2022 : When Personalization Harms: Reconsidering the Use of Group Attributes of Prediction »
Vinith Suriyakumar · Marzyeh Ghassemi · Berk Ustun -
2022 : Real world relevance of generative counterfactual explanations »
Swami Sankaranarayanan · Thomas Hartvigsen · Lauren Oakden-Rayner · Marzyeh Ghassemi · Phillip Isola -
2022 : Just Following AI Orders: When Unbiased People Are Influenced By Biased AI »
Hammaad Adam · Aparna Balagopalan · Emily Alsentzer · Fotini Christia · Marzyeh Ghassemi -
2023 Affinity Workshop: Muslims in ML »
Sanae Lotfi · Hammaad Adam · Marzyeh Ghassemi · Shakir Mohamed · S. M. Ali Eslami -
2022 : Dissecting In-the-Wild Stress from Multimodal Sensor Data »
Sujay Nagaraj · Thomas Hartvigsen · Adrian Boch · Luca Foschini · Marzyeh Ghassemi · Sarah Goodday · Stephen Friend · Anna Goldenberg -
2022 : Just Following AI Orders: When Unbiased People Are Influenced By Biased AI »
Hammaad Adam · Aparna Balagopalan · Emily Alsentzer · Fotini Christia · Marzyeh Ghassemi -
2022 : Unsupervised Deep Metric Learning for the inference of hemodynamic value with Electrocardiogram signals »
Hyewon Jeong · Marzyeh Ghassemi · Collin Stultz -
2022 : Unsupervised Deep Metric Learning for the inference of hemodynamic value with Electrocardiogram signals »
Hyewon Jeong · Marzyeh Ghassemi · Collin Stultz -
2022 : Fair Multimodal Checklists for Interpretable Clinical Time Series Prediction »
Qixuan Jin · Haoran Zhang · Thomas Hartvigsen · Marzyeh Ghassemi -
2022 : Fair Multimodal Checklists for Interpretable Clinical Time Series Prediction »
Qixuan Jin · Haoran Zhang · Thomas Hartvigsen · Marzyeh Ghassemi -
2022 Workshop: Robustness in Sequence Modeling »
Nathan Ng · Haoran Zhang · Vinith Suriyakumar · Chantal Shaib · Kyunghyun Cho · Yixuan Li · Alice Oh · Marzyeh Ghassemi -
2022 Workshop: Learning from Time Series for Health »
Sana Tonekaboni · Thomas Hartvigsen · Satya Narayan Shukla · Gunnar Rätsch · Marzyeh Ghassemi · Anna Goldenberg -
2022 Poster: Semantic uncertainty intervals for disentangled latent spaces »
Swami Sankaranarayanan · Anastasios Angelopoulos · Stephen Bates · Yaniv Romano · Phillip Isola -
2022 Poster: If Influence Functions are the Answer, Then What is the Question? »
Juhan Bae · Nathan Ng · Alston Lo · Marzyeh Ghassemi · Roger Grosse -
2022 Poster: Robustness Analysis of Video-Language Models Against Visual and Language Perturbations »
Madeline Chantry · Shruti Vyas · Hamid Palangi · Yogesh Rawat · Vibhav Vineet -
2021 : Data Opportunities: unsolved medical problems and where new data can help »
Bin Yu · Regina Barzilay · Marzyeh Ghassemi · Emma Pierson -
2021 Workshop: Machine learning from ground truth: New medical imaging datasets for unsolved medical problems. »
Katy Haynes · Ziad Obermeyer · Emma Pierson · Marzyeh Ghassemi · Matthew Lungren · Sendhil Mullainathan · Matthew McDermott -
2021 Poster: Learning Optimal Predictive Checklists »
Haoran Zhang · Quaid Morris · Berk Ustun · Marzyeh Ghassemi -
2021 Poster: Characterizing Generalization under Out-Of-Distribution Shifts in Deep Metric Learning »
Timo Milbich · Karsten Roth · Samarth Sinha · Ludwig Schmidt · Marzyeh Ghassemi · Bjorn Ommer -
2021 Poster: Medical Dead-ends and Learning to Identify High-Risk States and Treatments »
Mehdi Fatemi · Taylor Killian · Jayakumar Subramanian · Marzyeh Ghassemi -
2020 : Policy Panel »
Roya Pakzad · Dia Kayyali · Marzyeh Ghassemi · Shakir Mohamed · Mohammad Norouzi · Ted Pedersen · Anver Emon · Abubakar Abid · Darren Byler · Samhaa R. El-Beltagy · Nayel Shafei · Mona Diab -
2020 Affinity Workshop: Muslims in ML »
Marzyeh Ghassemi · Mohammad Norouzi · Shakir Mohamed · Aya Salama · Tasmie Sarker -
2020 : Welcome »
Marzyeh Ghassemi -
2019 Poster: The Cells Out of Sample (COOS) dataset and benchmarks for measuring out-of-sample generalization of image classifiers »
Alex Lu · Amy Lu · Wiebke Schormann · Marzyeh Ghassemi · David Andrews · Alan Moses -
2018 Workshop: Machine Learning for Health (ML4H): Moving beyond supervised learning in healthcare »
Andrew Beam · Tristan Naumann · Marzyeh Ghassemi · Matthew McDermott · Madalina Fiterau · Irene Y Chen · Brett Beaulieu-Jones · Michael Hughes · Farah Shamout · Corey Chivers · Jaz Kandola · Alexandre Yahi · Samuel Finlayson · Bruno Jedynak · Peter Schulam · Natalia Antropova · Jason Fries · Adrian Dalca · Irene Chen -
2017 Workshop: Machine Learning for Health (ML4H) - What Parts of Healthcare are Ripe for Disruption by Machine Learning Right Now? »
Jason Fries · Alex Wiltschko · Andrew Beam · Isaac S Kohane · Jasper Snoek · Peter Schulam · Madalina Fiterau · David Kale · Rajesh Ranganath · Bruno Jedynak · Michael Hughes · Tristan Naumann · Natalia Antropova · Adrian Dalca · SHUBHI ASTHANA · Prateek Tandon · Jaz Kandola · Uri Shalit · Marzyeh Ghassemi · Tim Althoff · Alexander Ratner · Jumana Dakka -
2016 Workshop: Machine Learning for Health »
Uri Shalit · Marzyeh Ghassemi · Jason Fries · Rajesh Ranganath · Theofanis Karaletsos · David Kale · Peter Schulam · Madalina Fiterau