Natural Identifiers Enable Post-Hoc Privacy Audits for LLMs

Assessing the privacy of large language models (LLMs) is notoriously difficult, primarily because most existing auditing methods require specific "canary" data to be inserted during the model's training phase. This makes post-hoc auditing of already-trained models impractical and expensive, often necessitating full retraining. Furthermore, auditing whether a specific dataset was used for training (dataset inference) typically requires access to a private, non-member held-out dataset that shares the same distribution as the suspect data, which is frequently unavailable or hard to construct in real-world scenarios. To overcome these significant limitations, a new study proposes the use of "natural identifiers" (NIDs). NIDs are defined as structured random strings, such as cryptographic hashes or shortened URLs, that naturally appear within common LLM training datasets. The unique format of NIDs allows for the generation of an unlimited number of additional random strings that share the same distribution. These synthetically generated NIDs can then serve as alternative canaries for privacy audits or as same-distribution held-out data for dataset inference. The evaluation demonstrates that NIDs effectively facilitate post-hoc differential privacy auditing without any retraining and enable dataset inference for any suspect dataset containing NIDs, eliminating the need for a private non-member held-out dataset. This innovation significantly enhances the ability to conduct scalable and practical privacy assessments for LLMs.