single mention · model architecture
Routed-expert silencing test
A diagnostic test for MoE layers that removes the routed experts' output from a layer while keeping the shared expert.1 It measures how much routed experts contribute by comparing the resulting loss increase against alternatives such as randomly rerouted experts.2
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
silenced routed experts removes the routed path’s output and keeps the shared expert
— tech reportLoss increase from perturbing the routing of one MoE layer of Kolibri at step 225 000, all other layers untouched, on 65 536 held-out tokens against a baseline
— tech report
Also called silenced routed experts.
Filed under model architecture › channel mixer › mixture of experts