Normalized Low-Rank Adaptation

Paper recorded by Signals 4 on 2026-08-31 in cs.LG. Abstract reproduced from arXiv; link to the original below.

Published 2026-08-31 on arXiv · recorded by Signals 4 on 2026-09-01

Category: cs.LG · 机器学习 · first seen 2026-09-01

Abstract

While low-rank adaptation (LoRA) is widely used for parameter-efficient model adaptation, how to regularize its training dynamics for stable and effective optimization remains underexplored. Because LoRA initializes the up-projection to zero, its early optimization dynamics are largely governed by the down-projection. Building on this observation, we introduce Normalized Low-Rank Adaptation (NoRA)

Read on arXiv →