Dr. OPD: Learning What to Follow for Optimal On-Policy Distillation of Large Language Models
Paper recorded by Signals 4 on 2026-09-29 in cs.CL. Abstract reproduced from arXiv; link to the original below.
Published 2026-09-29 on arXiv · recorded by Signals 4 on 2026-09-30
Category: cs.CL · 自然语言处理 · first seen 2026-09-30
Abstract
On-policy distillation (OPD) trains a student on its own generated responses using dense, token-level supervision from a stronger teacher. Vanilla OPD treats all teacher signals equally, assuming that the teacher's supervision is equally important for every token. However, teacher signals at different tokens may have very different effects on the student's performance: some correct important reaso
Read on arXiv →
Cite this page: Dr. OPD: Learning What to Follow for Optimal On-Policy Distillation of Large Language Models: the #8 most recent of 292 cs.CL papers we have recorded (as of 2026-09-29). Source: Signals 4 (Signals API) — https://data.jiangzhang.ca/signals4/t/papers/dr-opd-learning-what-to-follow-for-optimal-on-policy-distillation-of-large-langu.html
Free to quote with attribution to “Signals 4 (Signals API)”. Machine-readable:
papers.json
Get 4 AI signals a day by email — free.
Get 4 AI signals a day by email — free