Paper recorded by Signals 4 on 2026-10-01 in cs.LG. Abstract reproduced from arXiv; link to the original below.
Published 2026-10-01 on arXiv · recorded by Signals 4 on 2026-10-02
Category: cs.LG · 机器学习 · first seen 2026-10-02
Multi-teacher on-policy distillation (MOPD) aims to combine the strengths of RL-trained teachers in a single student, but how teacher signals affect parameter changes remains underexplored. We study Qwen3-1.7B with four domain teachers trained with RL from the same initialization as the student, comparing gradients, optimizer updates, and task learning curves, with additional SmolLM3-3B diagnostic