Open Models Almanac
Methodsmodel architectureprediction head

implementation detail · filed under model architecture

MTP speculative decoding

The sources on this page only mention it in passing. See prediction head for the methods filed beside it.

sources
11
models
7
labs adopt it
6
strongest
core

Documented in

Google DeepMind
Tencent (Hunyuan)
Z.ai (Zhipu AI)

Evidence

23 quotes from 11 sources, one card per model, each opening with its strongest.

How sources treat it
optional 2used 17core 4

NVIDIA-Nemotron-3-Ultra-550B-A55B

NVIDIAadopted3 sources

Multi Token Prediction (MTP) leads to faster inference with speculative decoding

coretraining objectivein Nemotron 3 UltraNVIDIA
9 more from this model

Multi-Token Prediction for native speculative decoding

coremodel architecturein Nemotron 3 UltraNVIDIA

Head-only KL distillation aligns MTP drafts with backbone logits for faster speculative decoding.

usedpost trainingin Nemotron 3 UltraNVIDIA

Head-only KL distillation aligns MTP drafts with backbone logits for faster speculative decoding.

usedinference servingin Nemotron 3 UltraNVIDIA

Multi Token Prediction (MTP) leads to faster inference with speculative decoding

usedinference servingin Nemotron 3 UltraNVIDIA

Multi Token Prediction (MTP) leads to faster inference with speculative decoding

usedinference servingin Nemotron 3 UltraNVIDIA

Includes MTP layers for faster inference through native speculative decoding.

usedinference servingin Nemotron 3 UltraNVIDIA

Includes MTP layers for faster inference through native speculative decoding.

usedinference servingin Nemotron 3 UltraNVIDIA

we use speculative decoding with Multi-Token Prediction (MTP).

usedinference servingin Nemotron 3 UltraNVIDIA

Multi-Token Prediction for native speculative decoding

usedinference servingin Nemotron 3 UltraNVIDIA

MiMo-V2.5

Xiaomiadopted3 sources

With one MTP layer for speculative decoding, the acceptance rate is about 90%.

usedinference servingin MiMo-7B seriesXiaomi
4 more from this model

by repurposing MTP as a draft model for speculative decoding

usedinference servingin MiMo-V2-FlashXiaomi

by repurposing MTP as a draft model for speculative decoding

usedinference servingin MiMo-V2-FlashXiaomi

A 3-layer Multi-Token Prediction (MTP) head is included for speculative decoding.

usedmodel architecturein MiMo-V2.5Xiaomi

A 3-layer Multi-Token Prediction (MTP) head is included for speculative decoding.

usedinference servingin MiMo-V2.5Xiaomi

GLM-5.2

Z.ai (Zhipu AI)adopted2 sources

We also improve GLM-5.2’s MTP layer for speculative decoding, increasing the acceptance length by up to 20%

usedinference servingin GLM-5.2Z.ai
3 more from this model

We also improve GLM-5.2’s MTP layer for speculative decoding, increasing the acceptance length by up to 20%

usedinference servingin GLM-5.2Z.ai

improve GLM-5.2’s MTP layer for speculative decoding, increasing the acceptance length by up to 20%

usedmodel architecturein GLM-5.2Z.ai

MTP layer for speculative decoding

usedinference servingin GLM-5.2Z.ai

Step-3.7-Flash

StepFunadopted1 source

supports vLLM speculative decoding with: --speculative-config '{"method": "mtp", "num_speculative_tokens": 3}'

optionalinference servingin Step 3.7 Flash NVFP4StepFun
↑