Low-Bit GDN Recurrent-State Quantization: Global Gradient-aware Allocation and Pure-Write Sliding Windows

Paper recorded by Signals 4 on 2026-10-29 in cs.LG. Abstract reproduced from arXiv; link to the original below.

Published 2026-10-29 on arXiv · recorded by Signals 4 on 2026-10-30

Category: cs.LG · 机器学习 · first seen 2026-10-30

Abstract

Linear attention stores historical information in a recurrent state of fixed size, providing a compact memory representation for long-context inference. Reducing state precision further lowers per-request storage requirements, but quantized writeback errors enter subsequent recurrences, degrade prediction quality on long-reasoning tasks, and can be accompanied by repetitive generation and excessiv

Read on arXiv →