Predictive Latents for Autoregressive Language Models: Possibility, Gauge, and Obstruction

Paper recorded by Signals 4 on 2026-10-20 in cs.CV. Abstract reproduced from arXiv; link to the original below.

Published 2026-10-20 on arXiv · recorded by Signals 4 on 2026-10-21

Category: cs.CV · 计算机视觉 · first seen 2026-10-21

Abstract

Autoregressive language models are trained by next-token cross-entropy, yet several recent systems also iterate a hidden state without emitting a token: continuous thoughts, pause steps, and recurrent depth. This note separates three risks on one latent process. The token risk $R_{\mathrm{tok}}$ scores a readout. The teacher-forced latent risk $R_1$ scores a one-step map that sees the true hidden

Read on arXiv →