Open Models Almanac
Methodsmodel architecturechannel mixermixture of experts

single mention · model architecture

Routed-expert silencing test

A diagnostic test for MoE layers that removes the routed experts' output from a layer while keeping the shared expert.1 It measures how much routed experts contribute by comparing the resulting loss increase against alternatives such as randomly rerouted experts.2

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. silenced routed experts removes the routed path’s output and keeps the shared expert — tech report
  2. Loss increase from perturbing the routing of one MoE layer of Kolibri at step 225 000, all other layers untouched, on 65 536 held-out tokens against a baseline — tech report

Also called silenced routed experts.

Filed under model architecture › channel mixer › mixture of experts