Quantifying memorization and memorization auditing: use canaries and exposure to measure how much I 'memorized' before release
In one sentence insert random canaries into the training set, then use exposure to measure how strongly I prefer each one over random strings — higher exposure means it's memorized harder. Quantifying Memorization (ICLR 2023) goes further and measures audit memorization with canaries + exposure before release, turning "how much got memorized" into a regression-able number — don't say "it probably didn't memorize anything" on a hunch, which is the false security of a missing audit.