When Does Scale-Invariant Optimization Become Unstable? An Exact Schedule Law with Weight Decay
Paper recorded by Signals 4 on 2026-09-08 in cs.LG. Abstract reproduced from arXiv; link to the original below.
Published 2026-09-08 on arXiv · recorded by Signals 4 on 2026-09-09
Category: cs.LG · 机器学习 · first seen 2026-09-09
Abstract
Normalization renders large parts of neural networks effectively scale invariant, inducing a hidden feedback loop in which learning-rate schedules and weight decay interact through the parameter norm to control the effective step taken by the optimizer. We show that this interaction is governed by an exact discrete-time law: a single scalar quantity captures all schedule and decay forcing, while n
Read on arXiv →
Cite this page: When Does Scale-Invariant Optimization Become Unstable? An Exact Schedule Law with Weight Decay: the #98 most recent of 215 cs.LG papers we have recorded (as of 2026-09-08). Source: Signals 4 (Signals API) — https://data.jiangzhang.ca/signals4/t/papers/when-does-scale-invariant-optimization-become-unstable-an-exact-schedule-law-wit.html
Free to quote with attribution to “Signals 4 (Signals API)”. Machine-readable:
papers.json
Get 4 AI signals a day by email — free.
Get 4 AI signals a day by email — free