There Is No Free Loop: When Does Looping a Trained Model Pay for Its Inference Compute?

Paper recorded by Signals 4 on 2026-10-30 in cs.AI. Abstract reproduced from arXiv; link to the original below.

Published 2026-10-30 on arXiv · recorded by Signals 4 on 2026-11-02

Category: cs.AI · 人工智能 · first seen 2026-11-02

Abstract

Looping re-applies a block of layers so that each token passes through the block more than once. Adding a loop to a trained model is attractive because it leaves the parameter count, and hence the disk size, the memory footprint and the training cost unchanged. Unfortunately, every generated token pays for the loop. Re-applying m of N layers makes each token about (N+m)/N times as expensive. Loops

Read on arXiv →