Training-data extraction
In one sentence the larger the model, the more often it was duplicated, and the more context you give, the more discoverable memorization there is.
Quantifying memorization & auditing
In one sentence insert random canaries into the training set, then use exposure to measure how strongly I prefer each one over random strings — higher exposure means it's memorized harder. Quantifying Memorization (ICLR 2023) goes further and measures audit memorization with canaries + exposure before release, turning "how much got memorized" into a regression-able number — don't say "it probably didn't memorize anything" on a hunch, which is the false security of a missing audit.
Training-data deduplication
In one sentence dedup is a high-return move that cuts memorization / extraction / membership-inference risk a lot; but it lowers frequency and probability, not a formal guarantee — a rare sample appearing once can still be memorized; for a formal guarantee, stack DP.
Privacy-targeted poisoning
In one sentence by buying expired domains for a "split-view," you can control ~0.01% of LAION-400M for ~\$60 (Carlini et al., S&P 2024 — this is a feasibility result, not itself a privacy-leak result). Conclusion first: training-data integrity = a privacy problem; an untrusted data source is a privacy amplifier.
DP privacy auditing
In one sentence insert many independent probes into training, then use how well you can guess "which probes were trained on" to derive an empirical ε lower bound — the privacy you actually deliver is no stronger than that bound. Steinke et al. (NeurIPS 2023 Outstanding Paper) made it runnable in one training run, cheap enough to keep as a regression check. Conclusion first: "we used a DP library" does not mean "the ε holds", and auditing is the only empirical way to turn a claimed ε into an audited ε.
Multimodal image extraction
In one sentence don't assume "a generative model only paints new images, it won't emit the originals." Image-domain memorization is real, the defenses share the same root as text memorization (dedup + DP + similarity auditing), but the cost is now "near-copy" rather than "verbatim."