single mention · model architecture
Architecture-aware MTP-head initialization by projection fusion and pruning
Initializes an MTP head by fusing latent MoE projections into expert projections and pruning the hidden size.
The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See prediction head for the methods filed beside it.
Filed under model architecture › prediction head