Machine Unlearning with a Destination: Tracking a Minimizer Path to the Retain-Only Objective

Many approximate machine-unlearning methods reduce the likelihood assigned to forget-set examples but lack a principled stopping point, making the resulting model sensitive to how long the unlearning procedure is run. We propose $\texttt{COAST}$ (Continuation via Objective Annealing and Secant Tracking), which uses secant prediction and gradient correction to track a path of minimizers from the full-data to the retain-only objective. Under local strong convexity and regular objective derivatives, we prove a uniform $O(T^{-2})$ tracking guarantee with a fixed correction budget, whereas omitting secant prediction under the same correction budget yields an $O(T^{-1})$ rate. Experiments with Phi-1.5 on TOFU and 7B models on MUSE show that $\texttt{COAST}$ approaches models retrained using only the retain data with small correction budgets.