Predictive Coding Networks Show Unique Topological Simplification

A new research paper investigates the topological properties of learned representations within Predictive Coding Networks (PCNs), a neuro-inspired bidirectional architecture. Using quantitative layer-wise persistent homology analysis, the study examined how topological features evolve across layers in PCNs trained on synthetic classification and MNIST datasets, achieving high accuracy. The findings reveal that smaller PCNs tend to collapse connected components across layers earlier in their architecture compared to larger models, indicating a relationship between model capacity and topological simplification. Furthermore, a strong negative correlation was observed between the depth of simplification and reconstruction error, suggesting that models simplifying later achieve better reconstruction. Crucially, PCNs consistently demonstrated later topological simplification than matched Multi-Layer Perceptrons (MLPs), with an average difference of 3.6 layers. These results highlight that both model size and the recurrent, bidirectional dynamics inherent to predictive coding inference play a significant role in shaping the compression-reconstruction tradeoff within these networks.