Rethinking On-Policy Distillation of Large Language Models II: One Training Example

Paper recorded by Signals 4 on 2026-09-03 in cs.AI. Abstract reproduced from arXiv; link to the original below.

Published 2026-09-03 on arXiv · recorded by Signals 4 on 2026-09-04

Category: cs.AI · 人工智能 · first seen 2026-09-04

Abstract

On-policy distillation (OPD) combines student-generated rollouts with dense token-level supervision from a teacher. Existing work has mainly studied its algorithmic behavior, leaving the role of training data unclear. We examine this role at the data-minimal limit by training on a single query. One-shot OPD keeps improving for hundreds of steps and recovers most of full-data OPD's gain across task

Read on arXiv →