Paper recorded by Signals 4 on 2026-09-08 in cs.LG. Abstract reproduced from arXiv; link to the original below.
Published 2026-09-08 on arXiv · recorded by Signals 4 on 2026-09-09
Category: cs.LG · 机器学习 · first seen 2026-09-09
Recurrent models provide a natural path to long-context modeling, yet models trained with backpropagation through time (BPTT) often fail beyond their training horizon. Classical analyses emphasize gradients that vanish or explode along temporal paths. However, dense per-token losses can still train a shared recurrent rule despite severe decay, showing that decay alone does not determine whether le