Open Models Almanac
Methodsinference & servingserving parallelism

methods · inference & serving

serving parallelism

20 methods filed at this node or below it, from the sources of 19 models.

In this branch 20

Everything filed at this node or below it, with one collapsible heading per child node.

filed here 20

Tensor parallelism default · 4 sources · 4 quotes
Attention data parallelism used · 5 sources · 6 quotes
DeepEP used · 4 sources · 4 quotes
Prefill-decode disaggregation used · 4 sources · 4 quotes
Topology-aware NVLink domain placement used · 2 sources · 2 quotes
Disaggregated serving optional · 2 sources · 2 quotes
14 single mentions
KDA state and KV-cache layout pinning core · 1 source · 1 quote
Data-parallel vision encoding used · 1 source · 1 quote
Disaggregated serving across 4+4 GPUs used · 1 source · 1 quote
FP8 parameter quantization during transfer used · 1 source · 1 quote
Tensor and expert parallelism used · 1 source · 1 quote
Tensor parallelism for MoE layers used · 1 source · 1 quote
TensorRT-LLM all-reduce backend used · 1 source · 1 quote
Expert parallelism optional · 1 source · 1 quote
Low-precision MoE combine optional · 1 source · 1 quote
MoonEP token migration for load balancing unclear · 1 source · 1 quote

By model

Which of this branch's methods each model's own documents describe, and how strongly. Under each model: its strongest treatment anywhere in the branch.

Modelmethods
GLM-5.3-Flash coreKDA state and KV-cache layout pinning coreDisaggregated serving across encoding, prefill, and decode usedPrefill-decode disaggregation usedRound-robin routing for prefill-decode disaggregation used—
Hy4-preview usedTensor parallelism used—
Step-5-Preview mentionedAttention data parallelism mentioned—
Hy3 usedTensorRT-LLM all-reduce backend used—
MiMo-V2.6-Flash usedDeepEP used—
NVIDIA-Nemotron-3-Ultra-550B-A55B usedPrefill-decode disaggregation usedTopology-aware NVLink domain placement usedAttention data parallelism optionalLow-precision MoE combine optional—
GLM-5.2 usedAttention data parallelism usedPrefill-decode disaggregation used—
Inkling usedFused reduce-scatter/all-gather collectives used—
Kimi K3 coreZero-copy fused token permutation and unpermutation coreMoonEP token migration for load balancing unclear—
Kolibri-1 usedFP8 parameter quantization during transfer used—
MiMo-V2.5 usedDeepEP usedAttention data parallelism optional—
MiMo-V2.5-Pro optionalDeepEP optional—
MiMo-V2.6-Pro usedDeepEP used—
Mistral-Large-4 usedDisaggregated serving across 4+4 GPUs usedTensor and expert parallelism used—
NVIDIA-Nemotron-3.5-Lightning-30B-A3B evaluatedDisaggregated serving evaluated—
NVIDIA-Nemotron-3.5-Super-VL-120B-A12B optionalDisaggregated serving optional—
Qwen3.5-397B-A17B usedData-parallel vision encoding usedTensor parallelism optional—
Qwen3.8-Flash-Next defaultTensor parallelism defaultExpert parallelism optional—
gpt-oss-120b usedTensor parallelism for MoE layers used—
↑