Paper recorded by Signals 4 on 2026-09-30 in cs.AI. Abstract reproduced from arXiv; link to the original below.
Published 2026-09-30 on arXiv · recorded by Signals 4 on 2026-10-01
Category: cs.AI · 人工智能 · first seen 2026-10-01
Looped transformers and Mixture-of-Experts (MoE) offer complementary routes to efficient scaling: recurrence increases computational depth at fixed parameters, while MoE sparsity expands total capacity at fixed active compute. Yet existing scaling laws model recurrence or sparsity in isolation. In this work, we introduce Loop Scaling Laws, the first scaling law to jointly model recurrence and spar