New Differentiable Logic Networks Outperform Fixed-Connection Models

A new research paper details a novel approach for optimizing the connections within deep differentiable logic gate networks (LGNs) and lookup table networks (LUTNs). This method involves training a probability distribution over potential connections for each gate or lookup table input, simultaneously learning the optimal gate types or LUT entries. The study demonstrates that these connection-optimized LGNs surpass traditional fixed-connection LGNs in performance on datasets such as Yin-Yang, MNIST, and Fashion-MNIST. Crucially, they achieve this while requiring substantially fewer logic gates, with one-layer networks achieving high accuracy on MNIST with almost 50 times fewer gates than fixed-connection counterparts. The technique also ensures training stability for deeper networks and reduces trainable parameters in LUTNs while improving accuracy.