Paper recorded by Signals 4 on 2026-10-09 in cs.CL. Abstract reproduced from arXiv; link to the original below.
Published 2026-10-09 on arXiv · recorded by Signals 4 on 2026-10-12
Category: cs.CL · 自然语言处理 · first seen 2026-10-12
Long-context serving runs out of KV memory before the hardware reaches its useful throughput. HiSparse stores history in host memory and fetches only the selected entries, but it uses one cache budget at every load and does not handle draft-model speculative decoding such as DSpark and DFlash2. We present Adaptive DHiSparse, which supports speculative decoding over hierarchical sparse KV and adapt