New 3D Gaussian Splatting Method Improves Talking Head Lip Sync.

Creating realistic audio-driven talking heads has been a challenge, particularly in achieving accurate lip synchronization. Existing 3D Gaussian Splatting (3DGS) methods often struggle with over-smoothed mouth movements and fail to accurately represent discrete articulatory events like lip closures, leading to an unnatural "leaky mouth" effect. This issue arises because continuous acoustic data is typically regressed to infer brief, distinct mouth configurations, biasing predictions towards average positions. A new approach, Phoneme-Driven Gaussian Splatting (PD-GS), addresses this by integrating explicit, frame-aligned linguistic targets. It augments the 3DGS model with phoneme tokens derived from an automatic speech recognition and forced-alignment pipeline. The core innovation is the Linguistic Fusion Module (LFM), which intelligently blends continuous audio context with discrete phoneme embeddings. This learned gating mechanism allows the model to maintain fluid audio-driven dynamics while providing stronger phoneme-specific guidance for critical articulation segments. Trained solely on monocular video using image reconstruction and lip landmark supervision, PD-GS demonstrates superior lip geometry and significantly reduces closure violations in complex phoneme sequences. This results in more accurate and believable neural avatars, enhancing the realism of synthetic speech and video.