New Cost Accounting for Reactive Computational Graphs

Performing exhaustive interventions on neural network computational graphs, such as activation-patching sweeps or systematic ablation studies, involves mutating the graph at numerous points. The computational cost of these operations is primarily driven by the need to recompute parts of the graph after each mutation. This research offers a comprehensive cost accounting for such workloads, specifically within the context of a reactive graph engine that only recomputes the affected downstream nodes. The study reveals several key insights. Firstly, the aggregate speedup of an exhaustive sweep over independent full recomputations is not a fixed constant; it varies depending on how per-layer weight changes with depth, converging to different ratios (e.g., (q+2)/(q+1) or q+2) rather than a universal '2'. A practical implication is a predicted wall-clock speedup ceiling of about 1.79 until interpreter overhead is optimized. Secondly, the paper precisely quantifies the cost of a sequence of persistent mutations, showing how interleaved costs exceed isolated sums and how batched applications are order-independent and sub-additive. Thirdly, it proves a "backward-locality gap" for the backward pass, demonstrating that backpropagation on architectures without long skip connections can collapse the aggregate speedup to 1. All these theoretical identities were rigorously validated using NeuroDSL, a reactive graph engine built in Julia, confirming the predicted limits and behaviors across various cost profiles and insertion orders.