SparseKAN Compresses AI Models for Efficiency Across Hardware

Kolmogorov-Arnold Networks (KANs) are a type of neural network that uses learnable univariate functions instead of scalar weights, introducing a new source of redundancy. SparseKAN addresses this by offering a comprehensive compression strategy. It targets three key areas: the number of basis functions, the count of neurons, and the numerical precision of the model. The method employs hierarchical learnable gates and a differentiable active-cost objective to identify and prune less important components. This allows for the physical compaction of models into smaller, dense tensors, rather than just using sparse masks. Experiments show substantial reductions in parameters without accuracy loss and significant improvements in inference speed on various hardware platforms, including GPUs and FPGAs.