EMAN Grows Neural Network Capacity for Multi-Task Learning

Current multi-task learning (MTL) methods often rely on predefined network structures, such as hard sharing, multiple paths, or adaptive sharing, to manage capacity. These structures typically change based on task boundaries or conflict signals. A fundamental question arises: can a network begin with a single computational path and only expand by creating new, independent paths when there's clear and sustained evidence from the optimization process? The proposed framework, Emergent Modular Atomic Network (EMAN), addresses this by introducing an optimization-driven mechanism. EMAN identifies an antisymmetric growth direction through latent relative phases without immediately instantiating a second path. It continuously monitors multiple decision signals during training, translating local optimization evidence into a structural decision. Only after this certification process does EMAN materialize two equal-capacity independent paths. This adaptive approach allows the network to allocate shared and task-specific representation capacity dynamically, tailoring it to the varying requirements of different tasks. Experiments on controlled rank settings, PASCAL-Context, and NYUv2 datasets confirm EMAN's effectiveness, showing improved performance with competitive computational costs.