Residual Dominance Explains Last-Item Bias in Recommenders

Transformer-based sequential recommender systems, particularly those employing causal self-attention, frequently exhibit a strong dependence on the user's most recent interaction when making predictions. The underlying structural reasons for this "last-item reliance" within the model's representation have been unclear. Researchers combined prediction-time diagnostics with a norm-based analysis of the full attention block to investigate this phenomenon. They found that while self-attention mechanisms aggregate contextual information, the subsequent residual addition significantly biases the final representation towards contributions from the same position. This effect is termed "residual dominance." To validate this interpretation, the study used inference-time residual scaling as a controlled intervention. Adjusting the residual strength created a clear trade-off: stronger residuals increased last-item reliance, while weaker residuals improved the model's ability to correctly rank items even when the final interaction was not the sole indicator. This work provides a structural explanation for the observed last-item bias in these recommender architectures.