New Neural Architecture Mimics Brain's Excitatory/Inhibitory Constraints

This research introduces a novel deep learning architecture designed to more closely mimic biological neural networks, specifically addressing Dale's constraint. This constraint dictates that biological neurons are either purely excitatory or purely inhibitory, and their synapses maintain a fixed sign. The proposed model uses two complementary non-negative channels, inspired by the brain's on-off representations, to handle positive and negative contributions.The architecture integrates these channels through a simple, repeating neural circuit motif across both bottom-up and top-down pathways. Coupled with a local Hebbian learning rule, the system propagates learning signals and updates weights using only local neuronal interactions. Theoretically, this approach can replicate backpropagation updates using only non-negative error signals.Empirically, the "on-off" architecture not only satisfies stricter biological constraints but also learns efficient representations, showing significant performance gains over comparable standard networks on benchmarks like Tiny ImageNet. These findings suggest that effective learning is achievable through biologically plausible mechanisms, moving towards more realistic models of neural computation.