Open Models Almanac
Methodsinference & servingserving parallelism

specific method · filed under inference & serving

Disaggregated serving

Separates serving work across distinct pools or workers rather than keeping it colocated; the evidence includes prefill/decode separation.

The sources on this page name it without explaining how it works. The line above is the extraction's one-line gloss. See serving parallelism for the methods filed beside it.

Also called Disaggregated inference serving, Disaggregated prefill and decode serving.

sources
2
models
2
lab adopt it
1
strongest
optional

Documented in

Evidence

2 quotes from 2 sources, one card per model, each opening with its strongest.

How sources treat it
evaluated 1optional 1

NVIDIA Dynamo supports distributed inference optimizations to increase throughput and reduce latency, including KV-cache-aware routing, KV cache management, and disaggregated serving.

optionalinference servingin NVIDIA DynamoNVIDIA
1 more from this model

The disaggregated deployment uses 7 GPUs for prefill only and 4 GPUs for decode only.

evaluatedinference servingin Nemotron 3.5 Super VLNVIDIA
↑