GRAFT Improves Zero-shot TTS Pronunciation with Reference Audio.

Researchers have developed GRAFT, a novel mechanism for text-to-speech (TTS) systems that enhances the fine-grained pronunciation of specific words. Current TTS models often struggle with unusual words, such as rare proper nouns or technical terms, due to the inherent ambiguity of text input. GRAFT addresses this by allowing users to provide a short audio sample of the desired pronunciation for a chosen word. This audio sample, encoded by the model's own speech tokenizer, is then bound to the word's position in the text prompt. A key innovation is the ability to disentangle the hint speaker's voice from the target speaker's voice during training, meaning the pronunciation hint can come from any voice, but the output speech maintains the desired target voice. Human evaluations and objective benchmarks confirm GRAFT's superior performance in rendering difficult words accurately while preserving naturalness and speaker similarity.