On-device inference as a privacy posture: the prompt really doesn't leave the device — but here's what "on-device" doesn't cover
In one sentence on-device means "the device doesn't send it out." But "on-device" is a word that's easy to over-read — the real boundary isn't "the vendor says on-device," it's which requests still fall back to the cloud, how much capability you trade away because the on-device model is smaller, and the telemetry / diagnostics / model-download channels. Bottom line first: "on-device" ≠ "nothing leaves the device." What's verifiable is that a given request completes on the local device and its prompt didn't leave the device for that path; but which requests run on-device versus fall back to the cloud is a deployment fact to check, not a model promise. For the fall-back-to-cloud portion, you go back to confidential inference to verify it.