Two $\operatorname{ReLU}$ Layers is All You Need

Paper recorded by Signals 4 on 2026-10-20 in cs.LG. Abstract reproduced from arXiv; link to the original below.

Published 2026-10-20 on arXiv · recorded by Signals 4 on 2026-10-21

Category: cs.LG · 机器学习 · first seen 2026-10-21

Abstract

We prove that every $\operatorname{ReLU}$ network can be exactly simulated by a $\operatorname{ReLU}$ network with two hidden layers. We present two proofs: one that is based on fillings in homological algebra and a direct combinatorial proof.

Read on arXiv →