Machine Unlearning with a Destination: Tracking a Minimizer Path to the Retain-Only Objective

Abstract

Many approximate machine-unlearning methods reduce the likelihood assigned to forget-set examples but lack a principled stopping point, making the resulting model sensitive to how long the unlearning procedure is run. We propose COAST (Continuation via Objective Annealing and Secant Tracking), which uses secant prediction and gradient correction to track a path of minimizers from the full-data to the retain-only objective. Under local strong convexity and regular objective derivatives, we prove a uniform $O(T^{-2})$ tracking guarantee with a fixed correction budget, whereas omitting secant prediction under the same correction budget yields an $O(T^{-1})$ rate. Experiments with Phi-1.5 on TOFU and 7B models on MUSE show that COAST approaches models retrained using only the retain data with small correction budgets.

Chulhee Yun

Associate Professor

I am an Associate Professor at KAIST AI. I am interested in optimization and machine learning theory.