InTrain Offers Unified Zero-Cost Metric for Neural Architecture Search

A new theoretical proxy called Intrinsic Trainability (InTrain) has been developed to revolutionize zero-cost Neural Architecture Search (NAS). Traditional zero-cost proxies often rely on fragmented heuristics, failing to capture the fundamental concept of what makes a neural network architecture trainable. InTrain formalizes trainability as an architectural invariant, emerging from two synergistic components: geometric capacity and optimization resilience. Geometric capacity is measured by the participation ratio of the activation covariance eigenspectrum, indicating the effective dimensionality of representation manifolds. Optimization resilience is assessed through cumulative gradient health, which evaluates the robustness of backpropagation across the network's depth. These two dimensions are synthesized through a scale-invariant multiplicative coupling, hypothesized to be crucial for capturing their synergistic relationship. Extensive experiments on standard NAS benchmarks demonstrate that InTrain achieves ranking correlations comparable to state-of-the-art ensemble-based proxies and surpasses other single-metric methods, offering a more efficient way to discover high-performance networks without expensive training.