AI Learns Optimal Compression Rules for Network Traffic

A new method called Robust Entropy Clustering for Adaptive comPression (RECAP) has been introduced to automatically learn efficient compression rules for structured network traffic. Network packets often contain highly redundant header fields within a flow, and RECAP aims to identify these patterns to replace predictable fields with shorter codes. The process is broken down into two stages: first, an unsupervised stage discovers underlying data structures by recursively partitioning training packets using an entropy-ratio criterion; second, a constrained selection stage employs dynamic programming to choose the optimal subset of rules that maximizes compression gain within a predefined budget for the number of rules. RECAP was specifically instantiated for Static Context Header Compression (SCHC), an IETF standard for constrained networks. Evaluations on real-world Internet-of-Things (IoT) and 5G core-network datasets demonstrated that RECAP significantly surpasses the performance of manually engineered rule sets. This method not only achieves better compression but also eliminates the need for laborious manual rule design, offering a more automated and efficient approach to network traffic compression.