Paper recorded by Signals 4 on 2026-11-02 in cs.LG. Abstract reproduced from arXiv; link to the original below.
Published 2026-11-02 on arXiv · recorded by Signals 4 on 2026-11-03
Category: cs.LG · 机器学习 · first seen 2026-11-03
In the near future language model pretraining will become bound by total data rather than total compute. Such a regime requires revisiting scaling laws and scaling law theory. Motivated by prior works that have found utility in ensembling, we study how the number of training epochs and ensemble size jointly shape generalization when training data are fixed. In language-model experiments on FineWeb