Universal Transformers for Circuit Computations: Perfect Length Generalization in Tiny Transformers

Paper recorded by Signals 4 on 2026-08-31 in cs.LG. Abstract reproduced from arXiv; link to the original below.

Published 2026-08-31 on arXiv · recorded by Signals 4 on 2026-09-01

Category: cs.LG · 机器学习 · first seen 2026-09-01

Abstract

Learning generalizable algorithmic computations remains a challenge for neural networks, as reflected in persistent failures on compositional and length generalization benchmarks. We present a provably correct, transformer parameterization (with only 280 learnable parameters for Boolean algebra tasks) capable of learning and evaluating problems of any depth or length. We assume inputs are fully pa

Read on arXiv →