Paper recorded by Signals 4 on 2026-09-09 in cs.CL. Abstract reproduced from arXiv; link to the original below.
Published 2026-09-09 on arXiv · recorded by Signals 4 on 2026-09-10
Category: cs.CL · 自然语言处理 · first seen 2026-09-10
LLM serving systems already reuse KV caches, but only when the reused text sits at the very start of the prompt. Two growing workloads break this condition: a retrieval-augmented generation server assembles a different set of retrieved chunks for every query, and a multi-agent coordinator reads reports written by other agents. Reused inside a new prompt, a cache carries the wrong positions and nev