CheckMIABench Provides Robust Benchmark for Language Model Membership Inference Attacks

Membership inference attacks (MIAs) are a standard method for evaluating the privacy properties of machine learning models. However, previous attempts to assess MIAs on large language models (LLMs) have faced challenges in establishing clean and statistically valid evaluations. A key issue has been subtle distribution shifts between member and non-member datasets, which can undermine the accuracy of MIA results, as demonstrated by "blind" methods outperforming published techniques on the same benchmarks. To address these difficulties, this paper introduces CheckMIABench, a new benchmark designed for principled evaluation of MIAs against LLMs. The core insight behind CheckMIABench is to utilize training data from before and after a fixed point during model training, ensuring that both member and non-member sets are drawn from the same underlying distribution. This approach allows any open-source model with intermediate checkpoints and public training data to be converted into a reliable MIA testbed. The framework has been applied to evaluate half a dozen published attacks on various Pythia and OLMo family models, ranging from 70M to 7B parameters. To further support privacy research, the authors have open-sourced a modular library that facilitates the design and implementation of attacks within this robust setting.