Open Models Almanac
Methodsmodel architecturechannel mixerdense feed-forward network

single mention · model architecture

Dense feed-forward network

A non-MoE feed-forward block, distinguished here from expert-routed MoE blocks.

The sources on this page name it without explaining how it works. The line above is the extraction's one-line gloss. See dense feed-forward network for the methods filed beside it.

Also called dense Feed-Forward Network (FFN).

Filed under model architecture › channel mixer › dense feed-forward network