Non-Robust Features Drive Training Data Privacy, Not Memorization

A new study redefines the understanding of how training data privacy is compromised in machine learning models. Contrary to the common assumption that data memorization or information dependency leads to vulnerability against image reconstruction attacks, the researchers found that privacy issues stem from "non-robust" features. These features, while generalizable, are imperceptible and unstable, and their presence dictates the model's susceptibility to data exposure. The paper presents evidence that even models with high memorization can be robust to reconstruction, and models trained with minimal pixel exposure can still be vulnerable. To address this, the authors propose Anti Adversarial Training (AT-AT), a novel method that intentionally trains models to learn these non-robust features. This approach not only enhances reconstruction defense but also improves accuracy, suggesting a new privacy-robustness trade-off.