CutClean Prunes Neural Networks for Enhanced Privacy

This paper introduces CutClean, a novel method for pruning neural networks that specifically aims to enhance privacy during inference. The core idea is to reduce the amount of sensitive information that flows through the network, even in scenarios where traditional dataset biases are not present. This is crucial for applications where models process sensitive attributes, as privacy leakage can occur regardless of representation imbalances. CutClean achieves this by employing auxiliary linear "privacy heads" at each block of the network. These heads are used to quantify the information leakage related to private attributes. The method then applies increasing levels of sparsity, effectively removing connections that contribute to this leakage, as measured by the accuracy of the privacy head attached to the final block. Experimental results on both synthetic and real-world datasets demonstrate that CutClean successfully minimizes the flow of private information. Crucially, it does so while simultaneously achieving high sparsity rates and preserving the model's accuracy on its primary classification task. This offers a promising approach for deploying neural networks in privacy-sensitive environments.