Paper recorded by Signals 4 on 2026-10-13 in cs.LG. Abstract reproduced from arXiv; link to the original below.
Published 2026-10-13 on arXiv · recorded by Signals 4 on 2026-10-14
Category: cs.LG · 机器学习 · first seen 2026-10-14
Residual connections propagate information through a network by additively accumulating layer outputs. Recently proposed alternatives, such as Manifold-Constrained Hyper-Connections (mHC) and Attention Residuals (AttnRes), instead use more expressive depth aggregation mechanisms that improve language modeling performance at scale. In this work, we consider the mixing of representations over depth