Paper recorded by Signals 4 on 2026-10-02 in cs.LG. Abstract reproduced from arXiv; link to the original below.
Published 2026-10-02 on arXiv · recorded by Signals 4 on 2026-10-05
Category: cs.LG · 机器学习 · first seen 2026-10-05
Neural network loss landscapes have many symmetries, which are preserved by gradient flow but broken by finite-stepsize stochastic gradient descent (SGD). A canonical example of such a symmetry is scale in homogeneous networks: one can scale up the parameters in one layer and down in the next without changing the network output. Previous work has documented cases in which SGD breaks this symmetry