Hebbian Learning Optimizes Neural Representation Under Biological Constraints

This study investigates whether a specific type of excitatory competitive Hebbian learning rule can effectively manage synaptic resource allocation within neural networks, considering biological limitations like costly synaptic maintenance and restricted connectivity. The goal is to determine if this approach can foster neural codes that compress relevant information into more efficient, low-redundancy patterns. Using mutual-information-based measures derived from the Variational Information Bottleneck, the researchers quantified representational cost. They conducted experiments with fixed audiovisual embeddings from three benchmarks (AVE, Kinetics-Sounds, VGGSound100) to focus on downstream associative plasticity. Hebbian learning was benchmarked against Dense Difference Target Propagation (DDTP) and backpropagation (BP) under identical sparsity and architectural constraints. The findings indicate that Hebbian learning achieves a lower task-information cost (CTI) than sparse BP and DDTP in compressed comparisons, and comparable CTI values to shallow BP with non-negative weights. Rather than simply boosting classification performance, Hebbian learning alters the balance between task-relevant information and representational cost, resulting in lower CTI at similar functional performance levels in several scenarios. This suggests Hebbian learning acts as a mechanism for efficient synaptic resource allocation rather than solely maximizing accuracy.