Open Models Almanac
Methodsmodel architectureprediction head

single mention · model architecture

Architecture-aware MTP-head initialization by projection fusion and pruning

Initializes an MTP head by fusing latent MoE projections into expert projections and pruning the hidden size.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See prediction head for the methods filed beside it.

Filed under model architecture › prediction head