Paper recorded by Signals 4 on 2026-09-28 in cs.LG. Abstract reproduced from arXiv; link to the original below.
Published 2026-09-28 on arXiv · recorded by Signals 4 on 2026-09-29
Category: cs.LG · 机器学习 · first seen 2026-09-29
The success of large language models (LLMs) has been accompanied by continued growth in model size and pretraining costs. Muon offers high accuracy and training efficiency in LLM pretraining. Recent work introduces row-wise normalization into Muon to balance update magnitudes and improve pretraining performance. However, row-wise normalization alone cannot accommodate different imbalance patterns