Paper recorded by Signals 4 on 2026-09-21 in cs.LG. Abstract reproduced from arXiv; link to the original below.
Published 2026-09-21 on arXiv · recorded by Signals 4 on 2026-09-22
Category: cs.LG · 机器学习 · first seen 2026-09-22
Linear RNNs based on the delta-rule enable efficient sequence modeling, but their linear updates with a low-rank correction constrain their expressivity. Prior work has shown that composing two delta-rule transitions in a single recurrent update can model a 2D rotation, but this increases the rank and the cost of the updates compared to a single transition. We show that Kimi Delta Attention (KDA)