Paper recorded by Signals 4 on 2026-10-09 in cs.AI. Abstract reproduced from arXiv; link to the original below.
Published 2026-10-09 on arXiv · recorded by Signals 4 on 2026-10-12
Category: cs.AI · 人工智能 · first seen 2026-10-12
Efficient attention mechanisms, such as linear and sliding window attention, aim to reduce the computational complexity of Transformers from quadratic to linear. However, their degraded in-context recall often necessitates interleaving full attention layers, leaving the quadratic bottleneck unresolved in long-context processing. To address this, we introduce Orthros attention, which shifts the eff