SemanticFold: Latent Sequence Compression SeparatesLanguage Modeling, Decodability, and Reasoning

Paper recorded by Signals 4 on 2026-10-07 in cs.CL. Abstract reproduced from arXiv; link to the original below.

Published 2026-10-07 on arXiv · recorded by Signals 4 on 2026-10-08

Category: cs.CL · 自然语言处理 · first seen 2026-10-08

Abstract

We study whether latent sequence compression of prompt prefixes preserves the capabilities that large language models rely on during inference. We introduce SemanticFold, a compression scheme that folds prefix hidden states at learned boundaries, and evaluate it across five model scales: Qwen3-1.7B, Qwen3-8B, SmolLM2-1.7B, Pythia-1.4B, and Pythia-6.9B. We use a fixed-target protocol: a frozen pref

Read on arXiv →