specific method · filed under inference & serving
Speculative sampling
A sampling-based speculative decoding method; the evidence names it but does not describe its specific mechanism.
- sources
- 3
- model
- 1
- lab adopt it
- 1
- strongest
- optional
How sources treat it
One count per evidence span, weakest treatment to strongest.
mentioned 2optional 1
Documented in
Further reading
Picked by hand, not extracted: where to read more, not evidence for anything on this page.
- Accelerating Large Language Model Decoding with Speculative Sampling (Chen et al., 2023) paper arxiv.orgintroduces speculative sampling
Evidence
3 spans quoted from the sources, strongest treatment first.
AngelSlim supports a comprehensive suite of compression tools ... including ... speculative sampling
optionalsoftware implementationin AngelSlimTencent Hunyuan
AngelSlim supports ... speculative sampling
mentionedinference servingin AngelSlimTencent
AngelSlim covers quantization, low-bit methods, and speculative sampling.
mentionedinference servingin AngelSlimTencent
Filed alongside
Other methods under inference & serving :: decoding strategy.
Speculative decodingMulti-Token PredictionDSparkEAGLEDFlashPresence PenaltyBest-of-N scaffoldingNEXTN speculative decodingMulti-layer EAGLEStandardized sampling configurationMulti-stage candidate filteringRecursive shared MTP-head draftingTask-specific sampling parametersChat Prefix CompletionConcurrency-aware draft-length tuningDistribution-matched draft-model fine-tuningEAGLE-3-style draft-model fine-tuningFused recurrent replay kernelGrammar-constrained decodingKV-cache sharing between drafter and targetLongest-trace selectionMTP-1 speculative decodingSame-checkpoint target and draft weightsThroughput-aware dynamic verification-length scheduling