Skip to yearly menu bar Skip to main content


Amazon

Expo Workshop

Forgetting on Purpose: A Hands-on Tutorial on LLM Unlearning

Zhao Gao ⋅ Jin Cheng ⋅ Diana Alvarado ⋅ Cher Simon

Grand BR B1
[ ]
Sun 6 Dec 2 p.m. AEDT — 5 p.m. AEDT

Abstract:

Large language models memorise personal data, copyrighted text and hazardous knowledge, and deleting training data alone does not remove it. This 3-hour tutorial introduces machine unlearning: removing specific knowledge from trained models while preserving their overall utility. It covers motivation, methods, evaluation, attacks and hands-on practice.

Agenda:
• Motivation & Background (15 min): why unlearning matters for privacy, copyright and safety; who can request removal; how it differs from fine-tuning and alignment.
• Foundations (15 min): the unlearning objective, exact vs. approximate unlearning, SISA, and tools for estimating data influence.
• LLM Unlearning Methods (25 min): gradient ascent, preference optimisation, locate-then-unlearn, representation misdirection, sparse autoencoders and parameter-efficient unlearning, and their trade-offs between forgetting, utility and cost.
• Vision & Multimodal Unlearning (15 min): concept erasure in diffusion models, and unlearning in multimodal LLMs and vision-language-action models.
• Break (15 min)
• Agent Unlearning (15 min): forgetting beyond weights, across memory, retrieval, tools, traces and the agent harness.
• Evaluation (20 min): what "forgotten" means in practice; TOFU, WMDP, MUSE, UnlearnCanvas and MLLMU-Bench; the limits of benchmark conclusions.
• Attacks & Robustness (20 min): whether forgetting survives fine-tuning, quantisation, probabilistic decoding, prompt attacks and concept resurgence.
• Hands-On Demo (30 min): a live notebook walking through the full loop of load, unlearn, measure, attack and compare.
• Open Problems & Discussion (10 min): open questions and audience Q&A.

Attendees will leave with a working map of the field, practical evaluation guidance and reusable code.

Chat is not available.