X-LogSMask Enhances Transformers for Graph Data.

Transformers have become ubiquitous architectures across various domains, but their inherent all-to-all self-attention mechanism is often ill-suited for graph-structured data, which typically features sparse, structured, and multi-scale interactions. Existing Graph Transformers attempt to bridge this gap through complex structural encodings, hybrid message-passing modules, or learned attention constraints, often at the cost of increased complexity and reduced interpretability. This research proposes X-LogSMask, an innovative and explainable multi-head logarithmic structural mask designed to directly integrate symmetrically normalized graph topology into the attention logits of a standard Transformer. The logarithmic transformation within X-LogSMask converts structural connectivity into a topology-aware gating signal. This signal effectively suppresses unsupported node interactions while preserving the feature-dependent aspects of attention. A key design choice is assigning different powers of the normalized adjacency matrix to different attention heads, which provides each head with a defined structural radius. This allows for multi-hop information propagation within a single Transformer layer, enhancing the model's ability to capture complex graph relationships. The paper further contextualizes a standard Transformer encoder as a one-step message-passing mechanism on a complete graph, thereby motivating X-LogSMask as a more topology-constrained and efficient alternative to unrestricted self-attention for graph data. Across 20 diverse node-, edge-, and graph-level benchmarks, Transformers equipped with X-LogSMask achieved state-of-the-art performance on 13 datasets and remained highly competitive even in a lightweight one-layer configuration. These results underscore that simple, interpretable structural masks can transform self-attention into a powerful graph-learning operator without requiring fundamental changes to the Transformer architecture.