Journey Operators Framework for Multi-Axis Data Composition.

Many types of data, such as sentences, images, or 3D volumes, possess inherent structure along one or more axes. While order is critical along a single axis (e.g., word order in a sentence), across independent axes, composition and movement should ideally be order-independent (e.g., moving right then down in an image yields the same result as down then right). This research develops a novel framework, "Journey Operators," to model these multi-axis structures. Each data item is associated with its content and a small transformation for each axis. A "journey operator," defined as the product of per-axis transformations along a path, governs both how data composes and how relative positions are described. The framework demonstrates that when these transformations are fixed, it naturally recovers established methods like Rotary Position Embedding (RoPE) and its multi-dimensional extensions. When transformations are data-dependent, the model gains a content-adaptive positional inductive bias. The paper rigorously proves the conditions under which these paths are well-defined and shows that, under specific symmetry and other assumptions, the resulting pairwise scoring rule must take the form of block-wise rotations, explaining the prevalence of RoPE-like methods. The theory is then used to design JoFormer, a model for value aggregation, and relates it to attention and state-space models.