Paper recorded by Signals 4 on 2026-10-05 in cs.CL. Abstract reproduced from arXiv; link to the original below.
Published 2026-10-05 on arXiv · recorded by Signals 4 on 2026-10-06
Category: cs.CL · 自然语言处理 · first seen 2026-10-06
Mixture-of-experts (MoE) routers typically apply softmax to the router scores and keep the top-K experts, making every token use exactly K experts. Sparsity-inducing probability maps such as sparsemax, alpha-entmax and normmax can adaptively assign exact zeros to selected experts, and therefore appear to offer token-dependent expert participation, even when using the same top-K machinery. In this