A Privileged Teacher That Only Scores Cannot Search: Critique-Conditioned On-Policy Self-Distillation for Reasoning

Paper recorded by Signals 4 on 2026-10-09 in cs.AI. Abstract reproduced from arXiv; link to the original below.

Published 2026-10-09 on arXiv · recorded by Signals 4 on 2026-10-12

Category: cs.AI · 人工智能 · first seen 2026-10-12

Abstract

On-policy self-distillation promises to replace the sparse outcome signal of verifiable-reward RL with dense token-level credit: a teacher conditioned on the reference solution scores the student's own rollout, and the resulting distributional shift is distilled back into the policy. We investigate whether this scoring signal actually contains corrective credit. Using gold error spans in student r

Read on arXiv →