Quantifying memorization and memorization auditing: use canaries and exposure to measure how much I 'memorized' before release
In one sentence insert random canaries into the training set, then use exposure to measure how strongly I prefer each one over random strings — higher exposure means it's memorized harder. Quantifying Memorization (ICLR 2023) goes further and measures audit memorization with canaries + exposure before release, turning "how much got memorized" into a regression-able number — don't say "it probably didn't memorize anything" on a hunch, which is the false security of a missing audit.
Unlearning benchmarks and standardized evaluation: turning "did it forget cleanly?" into a regression-able score — and the benchmarks' own blind spots
In one sentence not just "can it still recite it?" but forget quality × retained utility × privacy leakage under attack, three axes at once. And these benchmarks converge on an uncomfortable finding — few methods survive both the utility test and the leakage test (MUSE reports that of eight algorithms, only one avoids severe privacy leakage ⚠️ preprint). But the benchmarks have blind spots of their own: a benchmark is a proxy, and passing it ≠ truly forgotten (this picks up this volume's Unlearning verifiability); whatever it doesn't cover is a corner it can't see.