New Benchmark for Fast Domain Adaptation in Unsupervised Speech Units

Representation learning is a crucial technique for pretraining models or as a preliminary step in unsupervised speech modeling, often achieving strong performance. However, its effectiveness when encountering out-of-domain speech, particularly diverse accents, and how quickly it can adapt to new domains, remains less explored. This is especially critical for the long tail of non-standard accents. To address this gap, researchers have developed ABX-Accent, a new benchmark derived from the AESRC dataset. This benchmark includes speech data across 10 distinct English accents, providing a small, unlabeled training set for each accent (under 10 hours). It also adapts the Zero Resources Challenge ABX evaluation metrics to specifically assess performance on these accents. The study illustrates the benchmark's utility with a baseline model that fine-tunes a pretrained Contrastive Predictive Coding (CPC) model using adaptive domain normalization. This method, initially validated on LibriSpeech with a male/female split, achieved a substantial 23.6% relative improvement in across-speaker ABX scores on average when applied to the new accent benchmark, compared to models without adaptation. The data and metrics will be open-sourced.