Understanding Trade-offs in Multi-Task Learning Capacity.

This research explores the phenomenon of negative transfer in multi-task learning (MTL), reframing it not merely as an optimization artifact but as a direct consequence of insufficient shared model capacity and low task redundancy. The authors introduce a novel Capacity-Redundancy (CR) identity, which decomposes the total predictive information across tasks into joint predictive information (incorporating label redundancy via total correlation) and a residual coupling term that quantifies unresolved interference from the shared representation. The study presents two significant findings. First, it establishes a clustering-gap decomposition that provides necessary and sufficient conditions for when clustered sharing strategies will outperform global sharing approaches. Second, it develops a gradient-Total Correlation (TC) bridge within a Gaussian multi-task model, formally validating gradient cosine similarity as an effective proxy for ordering task redundancy. Empirical evaluations demonstrate that estimating the residual coupling from validation residual correlations shows that clustered LoRA (Low-Rank Adaptation) substantially reduces this coupling. This approach not only outperforms size-matched random partitions but also yields statistically significant gains, providing a clearer path to mitigating negative transfer and optimizing resource allocation in MTL.