Paper recorded by Signals 4 on 2026-10-08 in cs.CL. Abstract reproduced from arXiv; link to the original below.
Published 2026-10-08 on arXiv · recorded by Signals 4 on 2026-10-09
Category: cs.CL · 自然语言处理 · first seen 2026-10-09
While caching key-value (KV) states accelerates Large Language Model (LLM) decoding, this cache can dominate memory usage at long context lengths. One solution is to compress this memory by exploiting inter-layer cache similarities. However, most existing techniques necessitate architectural changes to LLMs and incur substantial overhead. In this work, we propose a symmetry-aware value cache mergi