Paper recorded by Signals 4 on 2026-11-01 in cs.LG. Abstract reproduced from arXiv; link to the original below.
Published 2026-11-01 on arXiv · recorded by Signals 4 on 2026-11-03
Category: cs.LG · 机器学习 · first seen 2026-11-03
Delta-rule linear attention underlies language models such as Kimi Linear and Qwen3-Next. Chunkwise implementations expose matrix parallelism, but their backward passes compute unnecessary auxiliary adjoints. We present ReWY, an algebraic reformulation that reduces this work before optimizing accelerator execution. Differentiating the chunk correction as one triangular residual system eliminates t