New Method Compresses Neural Networks with Minimal Realisation.

Deep neural networks often contain considerable redundancy within their hidden states, yet most compression techniques operate directly on weights, neurons, or quantized representations without explicitly characterizing the dynamic role of internal states. This research proposes a novel approach using a controllability-observability framework to achieve empirical state-order reduction in DNNs. The method treats a trained network as a depth-indexed nonlinear dynamical system. It constructs data-driven reachability, observability, and balanced Gramians from hidden-state snapshots and output Jacobians. These "A/B/C tests" then estimate layer-wise reachable, observable, and jointly reachable-observable ranks, which serve both as diagnostic measures of redundancy and as actual compressed layer widths for building reduced networks. Experiments on MNIST and CIFAR-10 datasets demonstrate significant compression. For instance, a four-layer SiLU DNN on MNIST was reduced by 72.95% in state and 73.48% in parameters with minimal accuracy loss. On CIFAR-10, a larger SiLU DNN achieved 70.94% state and 83.09% parameter compression, maintaining accuracy and reducing CUDA inference latency by approximately 3X. This shows that balanced reachable-observable ranks offer a principled empirical minimal-realization criterion for designing compact neural architectures.