Paper recorded by Signals 4 on 2026-11-01 in cs.LG. Abstract reproduced from arXiv; link to the original below.
Published 2026-11-01 on arXiv · recorded by Signals 4 on 2026-11-03
Category: cs.LG · 机器学习 · first seen 2026-11-03
We study the implicit bias of Adam and its similarity to sign gradient descent on homogeneous models in binary classification tasks. Perhaps surprisingly, we uncover that these depend on the learning rate schedule. While previous works studied strongly decaying learning rate schedules $η(t)=\mathrm o \left(t^{1/L-1}\right)$, where $L$ is the homogeneity degree, and showed $\ell_\infty$-margin-maxi