Turing LLM Post-Training: Calibrated Distillation and Efficient Dynamic-MoE Execution

Paper recorded by Signals 4 on 2026-10-29 in cs.LG. Abstract reproduced from arXiv; link to the original below.

Published 2026-10-29 on arXiv · recorded by Signals 4 on 2026-10-30

Category: cs.LG · 机器学习 · first seen 2026-10-30

Abstract

We study post-training for dynamically routed mixture-of-experts (MoE) models, focusing on integrating independently optimized specialists while keeping consolidation efficient. Our approach combines calibrated multi-teacher on-policy distillation (C-MOPD), which matches each selected teacher on a local Top-$K$ support with teacher-wise loss normalization and clipping, with an execution path that

Read on arXiv →