Paper recorded by Signals 4 on 2026-09-29 in cs.LG. Abstract reproduced from arXiv; link to the original below.
Published 2026-09-29 on arXiv · recorded by Signals 4 on 2026-09-30
Category: cs.LG · 机器学习 · first seen 2026-09-30
KV cache memory and bandwidth costs grow with context length and batch size, which limits efficient long-context inference. To address this bottleneck, we introduce WUSH-KV for low-bit KV-cache quantization. It adapts WUSH, which constructs a data-aware transform from the second-order statistics of both factors in a matrix product to reduce quantization error. WUSH-KV uses calibration data to cons