Training Communication-Efficient Mixture-of-Experts Language Models with Layer Re-Configuration
Paper recorded by Signals 4 on 2026-08-28 in cs.AI. Abstract reproduced from arXiv; link to the original below.
Published 2026-08-28 on arXiv · recorded by Signals 4 on 2026-08-31
Category: cs.AI · 人工智能 · first seen 2026-08-31
Abstract
When training Mixture-of-Experts (MoE) language models with expert parallelism, all-to-all token dispatch and combine collectives can consume a substantial fraction of end-to-end training time. In this work, we study communication-efficient MoE models (CE-MoE), in which we adopt a heterogeneous layer pattern that decouples token-mixing and channel-mixing depth. Compared to conventional models whic
Read on arXiv →
Cite this page: Training Communication-Efficient Mixture-of-Experts Language Models with Layer Re-Configuration: the #291 most recent of 300 cs.AI papers we have recorded (as of 2026-08-28). Source: Signals 4 (Signals API) — https://data.jiangzhang.ca/signals4/t/papers/training-communication-efficient-mixture-of-experts-language-models-with-layer-r.html
Free to quote with attribution to “Signals 4 (Signals API)”. Machine-readable:
papers.json
Get 4 AI signals a day by email — free.
Get 4 AI signals a day by email — free