Open Models Almanac
Methodssoftware implementationinference engine

methods · software implementation

inference engine

19 methods filed at this node or below it, from the sources of 15 models.

In this branch 19

Everything filed at this node or below it, with one collapsible heading per child node.

filed here 19

vLLM used · 7 sources · 7 quotes
TensorRT-LLM used · 3 sources · 4 quotes
SGLang optional · 4 sources · 4 quotes
KTransformers optional · 2 sources · 2 quotes
Unsloth optional · 2 sources · 2 quotes
14 single mentions
One-GPU FP8 vLLM replicas core · 1 source · 1 quote
vLLM general-plugins entry point core · 1 source · 1 quote
vLLM plugin core · 1 source · 1 quote
Atlas inference library used · 1 source · 1 quote
litert-lm serve used · 1 source · 1 quote
llama.cpp used · 1 source · 1 quote
llm-d routing with Model Express startup used · 1 source · 1 quote
vLLM or SGLang serving used · 1 source · 1 quote
vLLM tensor parallelism across eight GPUs used · 1 source · 1 quote
Dedicated serving engines optional · 1 source · 1 quote
vLLM language-model-only serving mode optional · 1 source · 1 quote
vLLM-Ascend optional · 1 source · 1 quote
xLLM optional · 1 source · 1 quote

By model

Which of this branch's methods each model's own documents describe, and how strongly. Under each model: its strongest treatment anywhere in the branch.

↑