Specialists Learn Faster: Knowledge Organization in Pre-training Shapes Knowledge Update Efficiency

Paper recorded by Signals 4 on 2026-10-09 in cs.LG. Abstract reproduced from arXiv; link to the original below.

Published 2026-10-09 on arXiv · recorded by Signals 4 on 2026-10-12

Category: cs.LG · 机器学习 · first seen 2026-10-12

Abstract

An ideal (language) model must continually update its knowledge. Existing work largely focuses on making these updates efficient after training. We investigate how the organization of time-varying knowledge during pre-training shapes the efficiency of subsequent updates. Models trained on chronologically ordered knowledge specialize in the most recently encountered knowledge; we call them speciali

Read on arXiv →