Multi Token Prediction (MTP) leads to faster inference with speculative decoding
9 more from this model
Multi-Token Prediction for native speculative decoding
Head-only KL distillation aligns MTP drafts with backbone logits for faster speculative decoding.
Head-only KL distillation aligns MTP drafts with backbone logits for faster speculative decoding.
Multi Token Prediction (MTP) leads to faster inference with speculative decoding
Multi Token Prediction (MTP) leads to faster inference with speculative decoding
Includes MTP layers for faster inference through native speculative decoding.
Includes MTP layers for faster inference through native speculative decoding.
we use speculative decoding with Multi-Token Prediction (MTP).
Multi-Token Prediction for native speculative decoding