Open Models Almanac
Methodssoftware implementationinference engine

specific method · filed under software implementation

vLLM

An inference engine used or recommended for model serving; the evidence also mentions serving recipes and hybrid-model disaggregation support.

The sources on this page name it without explaining how it works. The line above is the extraction's one-line gloss. See inference engine for the methods filed beside it.

Also called Serving MiMo-V2.6-Pro-RL with vLLM, vLLM model serving, serves the model with vLLM.

sources
7
models
5
labs adopt it
5
strongest
used

Documented in

Evidence

7 quotes from 7 sources, one card per model, each opening with its strongest.

How sources treat it
optional 4used 3

NVIDIA-Nemotron-3-Ultra-550B-A55B

NVIDIAadopted3 sources

disaggregation for hybrid Mamba-Attention models now works out-of-the-box in vLLM (Kwon et al., 2023)

usedsoftware implementationin Nemotron 3 Ultra
2 more from this model

disaggregation for hybrid Mamba-Attention models now works out-of-the-box in vLLM

usedsoftware implementationin Nemotron 3 UltraNVIDIA

Acceleration Engine: vLLM

usedsoftware implementationin Nemotron 3 UltraNVIDIA

MiniMax-M3

MiniMaxadopted1 source

We recommend the following inference frameworks to serve the model: vLLM

optionalsoftware implementationin MiniMax-M3MiniMax

Further reading

Picked by hand, not extracted: where to read more, not evidence for anything on this page.

↑