Paper recorded by Signals 4 on 2026-09-28 in cs.CL. Abstract reproduced from arXiv; link to the original below.
Published 2026-09-28 on arXiv · recorded by Signals 4 on 2026-09-29
Category: cs.CL · 自然语言处理 · first seen 2026-09-29
Attention often concentrates on a small subset of tokens in the context, but which subset matters changes from one query to the next. To exploit this changing structure, we introduce SANTA++, a training-free stochastic attention method that uses representative keys for memory-efficient selection without scanning the entire key-value (KV) cache. Cached keys are organized into teams, and the query s