Paper recorded by Signals 4 on 2026-09-08 in cs.CL. Abstract reproduced from arXiv; link to the original below.
Published 2026-09-08 on arXiv · recorded by Signals 4 on 2026-09-09
Category: cs.CL · 自然语言处理 · first seen 2026-09-09
Language-model checkpoints are commonly selected by pretraining loss or benchmark scores, assuming that the highest-scoring checkpoint will remain the best starting point for subsequent training. We show that this assumption can fail in a full 30B mixture-of-experts training pipeline. The checkpoints that perform better after the full downstream training stack also have higher solution density, i.