New Method Detects LLM Distillation Using Reference Models.

This paper introduces a novel method for detecting whether a large language model (LLM) has undergone distillation, a process where a smaller model is trained on outputs from a stronger, "teacher" model. While identifying a teacher from a student model in isolation is difficult, this research shows it becomes tractable when an earlier-generation checkpoint from the student's lineage is available as a reference. The proposed technique leverages reference-based membership inference, comparing how strongly a student model aligns with outputs from various candidate teachers relative to its own reference checkpoint. This allows for the identification of the most probable teacher and provides evidence of distillation. To address unknown distillation pipelines, the method infers proxy prompt templates directly from model outputs. It also identifies a unique glyph-level signal specific to certain models. Evaluating distillation detection is complex due to the entangled nature of modern model lineages. To overcome this, the authors developed a hybrid evaluation approach combining controlled distillation experiments with real-world models. In both scenarios, their method achieved near-perfect accuracy in single-teacher distillation cases, even with limited knowledge of the distillation process. The framework also includes statistical tests for teacher attribution and distillation detection, and it can be extended to open-world settings. Applying this to contemporary models, the research uncovered new evidence regarding potential distillation relationships involving models like QwQ, DeepSeek-R1, and GPT-OSS.