Late Attention Layers Alone Can Copy Entity Tokens, but Not Without Attending to Their Context

Paper recorded by Signals 4 on 2026-09-28 in cs.CL. Abstract reproduced from arXiv; link to the original below.

Published 2026-09-28 on arXiv · recorded by Signals 4 on 2026-09-29

Category: cs.CL · 自然语言处理 · first seen 2026-09-29

Abstract

Large language models (LLMs) reliably perform entity copying, in which a model copies tokens referring to an entity, termed entity tokens, from the prompt into its output to answer a question. Although entity copying is straightforward for most LLMs, existing research does not provide a systematic account of which layers specialize in this fundamental task or how other tokens in the same sequence,

Read on arXiv →