Discounted Least Squares Concentration Inequalities Found Flawed, Corrections Proposed.

Self-normalized concentration inequalities are fundamental tools in the theoretical analysis of bandit algorithms and reinforcement learning, particularly for understanding the behavior of estimators. A specific, widely adopted extension claims to provide a time-uniform guarantee for discounted least-squares estimators, which are crucial in non-stationary problems. This paper, however, uncovers a significant flaw in this claim. Through a simple scalar Gaussian counterexample, the researchers demonstrate that the asserted bounded radius of the inequality is violated with probability one. They further prove that for certain conditions, any valid deterministic anytime boundary must grow at least logarithmically with time, contradicting the original claim. The core of the error lies in the assumption that different terminal times use the same Gaussian mixing distributions, which prevents the fixed-time mixtures from forming a single supermartingale, thus invalidating the stopping-time argument. The paper not only identifies the proof error but also provides crucial corrections. It confirms the validity of the weighted inequality at fixed deterministic times and offers valid finite- and infinite-horizon corrections. These findings have significant consequences for the theoretical underpinnings of many existing analyses in reinforcement learning and bandit literature that rely on this flawed inequality.