single mention · inference & serving
MTP-1 speculative decoding
Uses MTP with a one-token speculative draft configuration for low-concurrency, latency-sensitive workloads.
Filed under inference & serving › decoding strategy
single mention · inference & serving
Uses MTP with a one-token speculative draft configuration for low-concurrency, latency-sensitive workloads.
Filed under inference & serving › decoding strategy