Skip to main content

3 docs tagged with "machine unlearning"

View all tags

Unlearning benchmarks and standardized evaluation: turning "did it forget cleanly?" into a regression-able score — and the benchmarks' own blind spots

In one sentence not just "can it still recite it?" but forget quality × retained utility × privacy leakage under attack, three axes at once. And these benchmarks converge on an uncomfortable finding — few methods survive both the utility test and the leakage test (MUSE reports that of eight algorithms, only one avoids severe privacy leakage ⚠️ preprint). But the benchmarks have blind spots of their own: a benchmark is a proxy, and passing it ≠ truly forgotten (this picks up this volume's Unlearning verifiability); whatever it doesn't cover is a corner it can't see.

Verifiability of machine unlearning: you deleted it but can't prove it — model-level unlearning is unverifiable, and "proofs" can be forged

In one sentence the same model parameters can be produced from a different dataset / a different gradient sequence, so a model owner can pass off a fabricated "I unlearned it" proof while actually keeping the record. Alongside it, TOFU (Maini et al., COLM 2024) turns "forget quality" into a measurable benchmark and finds no off-the-shelf method convincingly passes the "forget quality vs. utility" tradeoff. Conclusion first: audit the algorithm / process, not the final weights; "deleted" must be provable, or it's compliance theater.

Verifiable deletion and machine unlearning: "deleted the source record" ≠ "the model forgot" — and proving it forgot is harder still

In one sentence once memorization is in the weights, deleting the source doesn't make it vanish). Machine unlearning aims to make the model "behave as if it never saw that record." Two hard parts: ① how to actually forget (exact unlearning ≈ equivalent retraining, expensive; approximate unlearning is fast but unguaranteed); ② how to prove it forgot (verifiable deletion) — the piece engineering most lacks.