Continuous Audio Thinking Enhances Large Audio Language Model Performance

A novel framework called Continuous Audio Thinking (CoAT) has been introduced to significantly enhance the capabilities of large audio language models (LALMs). Traditionally, LALMs are optimized to produce text-aligned responses, which often results in the loss of rich acoustic information—such as phonetic nuances, prosody, sound events, affect, and pitch—as their internal states are progressively shaped for text generation. CoAT addresses this limitation by providing LALMs with a continuous latent workspace. Within this "thinking space," the model can effectively organize and retain diverse acoustic information. This is achieved through distillation from audio experts, which grounds the thinking space with high-fidelity acoustic knowledge. By leveraging this rich acoustic context, the model can generate more informed and accurate textual responses. A key advantage of CoAT is that its continuous thinking block can be processed in a single prefill step, meaning it does not incur additional autoregressive decoding costs compared to baseline models. The effectiveness of CoAT has been demonstrated across three prominent LALMs: Qwen2-Audio, Qwen2.5-Omni-7B, and Audio Flamingo 3. The framework yielded notable performance improvements on a broad suite of benchmarks, encompassing audio reasoning, general audio understanding, music classification, speech emotion recognition, and speech transcription. Further analysis confirms that the auxiliary supervision from the thinking positions successfully propagates to and enhances the model's textual outputs.