New Method Improves Deep Network Training Efficiency and Accuracy

Traditional deep learning relies on backpropagation, which requires non-local operations to route error signals. Predictive Coding (PC) offers a biologically inspired alternative, but still had one non-local dependency: the Jacobian-transpose product for error routing. This new work, WF-Act-PC, addresses this by showing how to compute this product locally for a specific class of deep network layers. The method achieves this by factoring the Jacobian transpose into three locally available terms, provided certain assumptions like weight symmetry and soft spectral-norm control are met. By restoring previously omitted corrections, WF-Act-PC closes the "transport gap" for these layers. Empirical results demonstrate that WF-Act-PC is the only PC method whose accuracy improves with depth, significantly outperforming existing PC baselines. It even matches or surpasses tuned backpropagation on deeper CIFAR-10 architectures and Tiny-ImageNet, making it a promising alternative for efficient and accurate deep network training.