New LLM Architecture Enables Native Unlearning of Specific Data Sources.

Large Language Models typically struggle with "unlearning" specific data because the contributions of different training sources are deeply intertwined. This new research introduces NULLs (Natively Unlearnable LLMs), an architecture designed to address this challenge by isolating source-specific information. It achieves this by training a shared "backbone" of neurons alongside a pool of sparsely activated "sinks" for each data source. During training, unique information from a source naturally concentrates in its dedicated sinks, while shared knowledge accumulates in the backbone. This structural separation allows for a source to be "unlearned" simply by disabling its corresponding sinks at deployment, eliminating the need for complex gradient updates or access to the original training data. The framework has been successfully scaled to large datasets like Wikipedia, demonstrating its ability to remove specific knowledge (e.g., a single article) while preserving related facts. It also shows robustness against adversarial extraction and relearning, suggesting that unlearning can be a built-in feature of LLMs from the outset.