Paper recorded by Signals 4 on 2026-10-15 in cs.CL. Abstract reproduced from arXiv; link to the original below.
Published 2026-10-15 on arXiv · recorded by Signals 4 on 2026-10-16
Category: cs.CL · 自然语言处理 · first seen 2026-10-16
On-policy distillation consolidates specialist models into one student at the price of rollouts, teacher passes and gradient steps for every teacher. We show that the consolidation does not need training: with forward passes only, the capabilities of separately tuned specialists can be aggregated into one model that is, on average, more accurate than either specialist alone. When specialists share