The essential 12
All 44 entries too many, and no idea where to start? Read these 12 first.
How they were picked (stated up front, so this doesn't become a subjective list): take the severity: high entries first, then rough-rank along two axes — engineers building LLM features hit it earliest, and its false security costs the most (once data crosses the boundary, it doesn't come back). Three entries are severity "medium" but belong to the governance and formal-defense basics you'll get wrong in your very first project if you skip them (data boundary, machine unlearning, DP) — they made the cut too. Finish these 12 and you've dodged the most common, most expensive privacy false-security beliefs.
To understand why these leaks happen, go to the Mechanism index; before shipping, walk the Minimum privacy baseline; for the real-world cost, read Real incidents & landmark demonstrations.
Handed out / into the weights — it doesn't come back
- "They don't train on it" is one cell of the data boundary — verify the terms item by item, date-stamp them, and they still change.
- Private data fed into my training can be pulled back out verbatim by an outside attacker — alignment and the chat wrapper won't stop it.
- Personal information in the training corpus resurfaces in my conversations — scrubbing reduces it, never eliminates it.
- Deleting the source data doesn't mean I forgot it — and "proving it's truly forgotten" is harder than forgetting.
You assumed anonymized / isolated — it isn't
- Turning text into vectors is not anonymization — vectors reconstruct back to the original text, even verbatim.
- If retrieval isn't filtered by the caller's permissions, your knowledge base is everyone's knowledge base.
- Shared memory / caches without per-user isolation will serve one person's data to another — a real incident already happened.
- Deciding "is this record in the training set" is the root of most privacy attacks — membership itself is sensitive.
The new plaintext surfaces of the agent & reasoning era
- A single instruction hidden in content can drive me to send private context out through a tool — zero-click exfiltration has hit mainstream products.
- System prompts / conversation context / tool results are not secrets — they can be coaxed out.
- My "thinking" isn't private — the more I reason, the more leaks.