Paper recorded by Signals 4 on 2026-09-23 in cs.LG. Abstract reproduced from arXiv; link to the original below.
Published 2026-09-23 on arXiv · recorded by Signals 4 on 2026-09-24
Category: cs.LG · 机器学习 · first seen 2026-09-24
Language models typically construct attention values from contextual hidden states, even when some of their content may be reusable across contexts. We investigate whether token-indexed memory can replace the dedicated value projection when complemented by contextual information. We propose Memory Attention (MA), which forms values by combining layer-specific token memory with contextual keys. The