DiffusionGemma Model Achieves Parallel Speech Recognition with Low WER

This research explores a novel approach to automatic speech recognition (ASR) by leveraging a discrete diffusion language model, DiffusionGemma, instead of traditional autoregressive decoders. The key innovation is the ability to refine an entire transcript in parallel over a small number of denoising steps, offering a departure from token-by-token generation. The system integrates a frozen Whisper encoder for acoustic features, a lightweight projector, and low-rank adapters to enable the frozen 26B mixture-of-experts backbone to process audio. A crucial finding was the need for a connectionist temporal classification (CTC) loss to properly ground the audio, as natural training objectives initially failed. The resulting model, trained with only 0.16% of the backbone's parameters, achieved a 6.6% word error rate on LibriSpeech test-clean and demonstrated efficient transcription in roughly eight parallel steps, regardless of utterance length, across multiple languages.