Paper recorded by Signals 4 on 2026-10-15 in cs.LG. Abstract reproduced from arXiv; link to the original below.
Published 2026-10-15 on arXiv · recorded by Signals 4 on 2026-10-16
Category: cs.LG · 机器学习 · first seen 2026-10-16
Reducing memory use in large language model (LLM) training is an ongoing challenge. Activation checkpointing saves memory by recomputation, while lossy compression changes the stored activations. The reemergence of sparse $\relu$ and $\relu^2$ activations creates a third opportunity: losslessly compressing the activation cache itself. We present BitSparse, a GPU-resident lossless compression forma