SQuAT: Spherical Lattice Vector Quantization-Aware Training

Paper recorded by Signals 4 on 2026-10-22 in cs.CL. Abstract reproduced from arXiv; link to the original below.

Published 2026-10-22 on arXiv · recorded by Signals 4 on 2026-10-23

Category: cs.CL · 自然语言处理 · first seen 2026-10-23

Abstract

Vector quantization (VQ), in particular lattice-based methods, offer strong rate-distortion trade-offs for low-bit large language models (LLMs), yet their behavior under quantization-aware training (VQ-QAT) remains poorly understood. We scale lattice VQ-QAT to billion-parameter LLMs and multi-billion-token training regimes, and introduce SQuAT, a shape-gain VQ-QAT approach trained with a separate

Read on arXiv →