expert-choice routing

Most MoE routers are token-choice: each token picks its top experts. Expert-choice flips the direction, each expert picks the top tokens it wants, up to a fixed capacity. Because every expert fills exactly its quota, load is balanced by construction: no expert is starved or overloaded, and you no longer need a separate balancing loss or token dropping.

Concretely, you compute the same token-by-expert affinity matrix, but instead of a per-token top-k over experts you take a per-expert top-c over tokens, where c is the capacity. A consequence is variable compute per token: a rich token may be selected by many experts while another is chosen by none. This is the opposite trade-off from token-choice and tends to give better load utilization and training throughput, at the cost of some tokens receiving little or no expert processing.

Expert-choice is attractive at training time, where perfect balance maximizes hardware utilization. It is awkward for autoregressive decoding, however, because the per-expert top-c must see a batch of tokens at once, a token's routing depends on its competitors, so causal one-token-at-a-time inference does not map cleanly onto it.