Sparse-Binary SOM Scales to 1 Million Neurons on Single GPU

Building large-scale Self-Organizing Maps (SOMs) for massive text corpora like MEDLINE has historically been impractical due to the computational demands of the best-matching-unit (BMU) search, which is bottlenecked by memory bandwidth. A new approach addresses this by introducing a feature-major codebook layout for sparse-binary SOMs. This innovative layout stores each feature's weights contiguously, transforming the BMU search into a tiled sparse-dense product. This change allows every loaded weight column to be reused across a tile of samples, significantly accelerating the search by 4.5 to 8.5 times without compromising quantization error. Paired with a radius-independent box-blur update and a convergence-based stopping rule, this method enables training a converged map over 29.9 million MEDLINE articles in about 72 seconds on a 24 GB GPU at 64x64 resolution. Crucially, it can accommodate 262,144 neurons (512x512 edges) on a 24 GB GPU, where other algorithms fail due to memory limits. On a 141 GB H200 GPU, it scales to an unprecedented 1,048,576 neurons (1024x1024 edges), making it the largest self-organizing map reported to date and demonstrating an 82x to 621x speedup over previous implementations.