Stability Annealing Guides Smoothed Sign Descent to Max-Margin Solutions

Adaptive gradient methods, widely used in machine learning, are known to exhibit different implicit biases compared to standard gradient descent, often leading to distinct max-margin separators in classification tasks. This research delves into the behavior of "stability-annealed smoothed-sign descent" for full-batch linear classification on separable data, focusing on how a controlled stability constant influences its optimization path. The study rigorously proves that, under specific conditions, the normalized iterates of this descent method converge to the minimizer of a convex Burg-type barrier function over a margin slice. This provides a precise mathematical characterization of the implicit bias, explaining *which* max-margin separator the algorithm selects. The proof involves re-framing the dynamics as entropic mirror ascent on a dual objective and controlling the dual gap. The findings include a full characterization of the static barrier geometry, its KKT conditions, and endpoint limits. Experimental validations confirm the theoretical dual identities and illustrate the predicted optimization path and rate diagram. This work sheds light on the fundamental mechanisms governing the implicit biases of certain adaptive optimization algorithms, particularly in the context of achieving robust classification boundaries.