Double-Scoring Improves Strong Lottery Ticket Extraction in Neural Networks

The lottery ticket hypothesis suggests that large neural networks contain smaller, sparse subnetworks capable of achieving similar performance. A "strong" version posits that these subnetworks can be accurate even before any training. While such strong lottery tickets are known to exist, reliably extracting them has been a challenge. This research revisits the edge-popup method, which uses frozen-weight score-training, and identifies layerwise sparsity selection as a key limitation. To address this, the paper proposes "double-scoring," an enhanced score-space parameterization. This technique replaces the problematic layerwise sparsity search with an optimization process over expanded score tensors. The authors prove that this augmented score space maintains access to all original-coordinate masks and can be interpreted as applying edge-popup to a zero-augmented network. Experimental results show that double-scoring substantially improves strong-ticket extraction compared to fixed-density edge-popup and pruning-at-initialization methods. It also surpasses rewound sparse-training topologies and exhibits reduced sensitivity to sparsity hyperparameters, indicating a more robust and effective approach to finding these powerful subnetworks.