Masked Distillation Internalizes LLM Chain-of-Thought Reasoning

Large Reasoning Models (LRMs) often generate lengthy, explicit intermediate steps, known as "chain-of-thought" (CoT), before arriving at a final answer. While these steps can aid in complex problem-solving, they significantly increase inference latency, memory usage, and serving costs. This research explores whether these computational steps can be internalized into the model's parameters, enabling it to produce answers more directly. The proposed solution is "masked distillation," a knowledge distillation framework. In this approach, a student LLM is trained to predict only the final solution tokens, conditioned on the original question. A reasoning teacher model provides feedback, having processed the question with its own CoT trace. This framework can be implemented in a self-distillation setting, where the same model acts as both teacher and student, or in a dual-model setting, with a larger teacher supervising a smaller student. Experiments on arithmetic and number-puzzle tasks demonstrate that this method allows the student to internalize reasoning, leading to shorter traces and reduced resource consumption while maintaining answer quality.