CAT-GS Improves Multimodal Learning Stability with Calibrated Gating

Training end-to-end multimodal neural networks often encounters stability issues, manifesting as three interconnected problems: one modality dominating optimization (imbalance), erratic modality selection due to noisy confidence cues (unstable gating), and conflicting gradients at the shared fusion layer (fusion interference). These issues degrade learning performance. A new optimization controller, CAT-GS (Calibrated, Adaptive, Thresholded Gating with Fusion Surgery), has been developed to stabilize these neural dynamics during backpropagation without altering model architectures or loss functions. CAT-GS employs temperature scaling and EMA smoothing to calibrate teacher-derived reliability, enabling a margin-thresholded policy to switch between warm-up dropout, weak-modality prioritization, and weak-biased blending. Furthermore, CAT-GS stabilizes gradient magnitudes under aggressive gating through capped gradient-budget renormalization and applies fusion-only PCGrad to mitigate destructive cross-modal interference at the primary shared bottleneck. Evaluated on various audio-visual and tri-modal benchmarks, CAT-GS consistently improves or matches fused multimodal accuracy against strong baselines, while also yielding smoother gating behavior and fewer conflicting fusion gradients.