New Virtual Node Design Enhances Message Passing Neural Networks.

Message-passing neural networks often use virtual nodes to facilitate global communication, but the conventional method compresses the entire graph into a single, homogeneous state, which is then broadcast uniformly to all nodes. This creates a bottleneck due to finite capacity. This research introduces an enhanced virtual node design aimed at alleviating this limitation without relying on self-attention mechanisms. The proposed design incorporates two key features: first, it factorizes global memory into independently writable and readable states using addressable cross-attention slots. Second, to preserve multiplicity—a challenge with standard softmax attention—each slot query is inserted as a private key/value anchor. This approach recovers discarded normalization mass, enabling an injective multiset representation capable of implementing a 1-WL refinement. Experimental results on tasks like multiplicity-aware Two-Radius analysis, motif counting, and constrained link-set prediction validate the effectiveness of this addressable and cardinality-preserving virtual memory, achieved at an arithmetic cost of O(nMd).