Nonequilibrium Phases of Repulsive Self-Attention: Chaos, Attention Condensation, and Emergent Locality

Paper recorded by Signals 4 on 2026-09-23 in cs.LG. Abstract reproduced from arXiv; link to the original below.

Published 2026-09-23 on arXiv · recorded by Signals 4 on 2026-09-24

Category: cs.LG · 机器学习 · first seen 2026-09-24

Abstract

We study the nonequilibrium dynamics of a minimal recurrent transformer with $N$ normalized tokens, $Q=K=I$, and a negative value map $V=-I$. Similarity-based attention selects nearby representations, while the negative value map drives tokens away from the selected field. This feedback can continually reorganize both the representation geometry and the attention network. For $d=2$, the tokens lie

Read on arXiv →