Stolen AI Models May Lack Practical Equivalence, Research Suggests

A recent research paper introduces a new perspective on model stealing attacks, arguing that simply achieving high fidelity in a surrogate model does not guarantee it will be functionally equivalent to the original service. The authors contend that because query-based extraction provides only partial information, many near-optimal surrogate models can exist, all with comparable fidelity but differing in other crucial deployment-relevant properties. Instead of focusing solely on classic learning-based model stealing, the research evaluates the 'Rashomon Set' of surrogate models, which is the collection of almost-equally-accurate models. By analyzing the diversity within this set using multiplicity and fairness metrics, the study reveals that surrogates, despite matching the target model's fidelity, can exhibit substantial variations in other critical performance aspects across various tasks and datasets. These findings suggest that the conventional wisdom regarding the economic leverage and practical equivalence of high-fidelity stolen models might be flawed. The inherent variability among seemingly similar models could diminish the real-world utility of such stolen intellectual property for adversaries.