Sparse Coding Compresses Vision-Language Model Latent Communication.

Vision-language models often communicate by exchanging continuous representations in a latent space, avoiding the need to serialize visual and reasoning states into text. While effective, current methods like Vision Wormhole transmit these messages as dense tensors of fixed size, regardless of the actual information content. This suggests a potential for significant redundancy and inefficiency in the communication channel. This research explores the compressibility of this latent communication by applying post-hoc sparse autoencoding to frozen Vision Wormhole activations. The goal was to determine if a more compact representation could be achieved without sacrificing performance. The findings indicate substantial compressibility: a sparse payload with only four active coefficients per token, using uint16 indices and float16 values, reduced transmitted bytes by an impressive 128 times. Crucially, this compression resulted in only a marginal change in mean accuracy across seven reasoning benchmarks (from 49.85% to 49.77%). This demonstrates that the communication channel between these agents is highly redundant and can be significantly optimized.