Open Models Almanac
Methodsmodel architecturechannel mixermixture of expertsexpert routing

single mention · model architecture

Sigmoid-based expert routing

Sigmoid-based expert routing is a token-to-expert routing scheme for mixture-of-experts layers that scores each token's affinity to experts with a sigmoid instead of softmax.1 It is motivated by avoiding the strong competition among experts that softmax normalization induces.2

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. we apply a sigmoid-activated function to the router logits — From Foundation to Application: Improving VLA Models in Practice
  2. To avoid the strong competition among experts induced by softmax normalization — From Foundation to Application: Improving VLA Models in Practice

Also called Sigmoid-based token-to-expert routing.

Filed under model architecture › channel mixer › mixture of experts › expert routing