Paper recorded by Signals 4 on 2026-09-16 in cs.AI. Abstract reproduced from arXiv; link to the original below.
Published 2026-09-16 on arXiv · recorded by Signals 4 on 2026-09-17
Category: cs.AI · 人工智能 · first seen 2026-09-17
Mixture-of-Experts (MoE) language models suffer from large parameter counts, which create a significant memory bottleneck. Expert pruning is the most direct approach for reducing this parameter count, yet existing methods make pruning decisions for each expert independently, and assume experts' contributions are purely additive. In reality, expert usage in MoEs is inherently cooperative. We derive