Knowledge Acquisition During Pre-training? Large Language Models Learn Better With Auxiliary Views

Paper recorded by Signals 4 on 2026-09-03 in cs.AI. Abstract reproduced from arXiv; link to the original below.

Published 2026-09-03 on arXiv · recorded by Signals 4 on 2026-09-04

Category: cs.AI · 人工智能 · first seen 2026-09-04

Abstract

Gaps remain in our understanding of how large language models (LLMs) acquire knowledge during pre-training. We posit that auxiliary views, reformulations of knowledge, are causally helpful for learning. We design controlled experiments to isolate this. First, we confirm that repetition is necessary for acquisition and clarify that paraphrasing helps only at smaller batch sizes. Second, holding the

Read on arXiv →