Do You Remember It All Too Well? An Information-Theoretic Taxonomy of Machine Unlearning

Machine unlearning has produced a heterogeneous family of algorithms, from gradient ascent on the forget set and retain-set fine-tuning to weight edits and influence approximations \citep{bourtoule2021machine, kurmanji2023towards, fan2024salun}. These are generally motivated by operational recipes rather than a shared definition of forgetting, and validated on benchmarks rather than against guarantees. It is therefore unclear what these methods optimise and when their notions of forgetting coincide. Motivated by this gap, we use the recent information-theoretic formulation of machine unlearning by \citet{xu2025machine} as a unifying framework for these algorithms. Let $Y$ denote the target variable for the prediction task, $S$ the released learning outcome produced by a map $f$, $S'$ the part of that outcome exposed to an outsider or auditor, and $Z$ the information to be forgotten. This framework formalizes forgetting as reducing the mutual information $I(S';Z)$. Hence, the unlearning objective is defined as $\min_f (1-\lambda)\,C(Y;S,Z) + \lambda\, I(S';Z)$, where $C(Y;S,Z)$ is a utility loss. This way, we trade the task utility against forgetting unwanted information. This objective provides the taxonomy structure that allows for methods localization and comparison. We organise unlearning methods along three axes derived from this framework, asking what each method *implicitly specifies*. The first is the **exposure variable** $S'$: forgetting may be enforced at the model input, an internal representation, parameters, or output. The second is the **forget signal $Z$**: this includes an observed feature or a latent data-inclusion label, the two settings of \citet{xu2025machine}, and a corruption signal, our extension incorporating corrective unlearning \citep{goel2024corrective}, where $Z$ is latent and only partially observed. The third is **how forgetting is enforced**: methods range from matching forgotten behaviour to a principled replacement distribution \citep{xu2025machine} to pushing the model away from its original behaviour, as in gradient ascent \citep{jang2022knowledge}. This taxonomy groups feature, data-point, and corrective unlearning as three choices of forget signal, rather than three separate problems. It also exposes combinations of choices that no surveyed method yet realises, which become concrete candidates to build. From this structure we can generate algorithmic hypotheses, testable by comparing utility--forgetting frontiers, information leakage, and distance from retraining. The taxonomy also identifies where formal guarantees from the theoretical literature might extend to modern deep-unlearning methods. Based on this taxonomy we propose new mathematical analysis of existing algorithms.