2 docs tagged with "model inversion"

In one sentence model inversion — using repeated queries + the confidence I emit to rebuild what a class's training sample "looks like" (Fredrikson et al. at CCS 2015 reconstructed recognizable faces from a face-recognition model); and attribute inference — given a person's partial known info + me, inferring their undisclosed sensitive attribute (Fredrikson et al.'s 2014 warfarin-dosing case). Conclusion first: confidence / probability outputs are the fuel, and attribute inference also borrows population-statistics correlations — don't assume "didn't send raw data out" is safe; look at output granularity, whether DP is stacked, and whether a class maps to a single individual.

In one sentence this is not private. The Feature-Space Hijacking Attack (Pasquini et al., CCS 2021) shows a malicious server can actively steer the split model into an insecure state and reconstruct the client's private training inputs from the intermediate activations (reconstructing images on MNIST / Omniglot / CelebA); UnSplit (Erdoğan et al., WPES @ CCS 2022) shows that even an honest-but-curious server — knowing only the client architecture, with no active interference — can invert (reconstruction MSE ≈ 0.08–0.15 on MNIST / Fashion-MNIST / CIFAR-10). Don't read "didn't send raw pixels" as privacy.