Techniquessoftware implementationinference engine

specific method · filed under software implementation

litert-lm serve

A CLI command for running a model as a local OpenAI-compatible API server.

source: 1
model: 1
lab adopt it: 1
strongest: used

How sources treat it

One count per evidence span, weakest treatment to strongest.

used 1

Documented in

Evidence

1 span quoted from the sources, strongest treatment first.

Run Gemma 4 12B as a local, OpenAI-compatible API server using the new litert-lm serve CLI command

usedsoftware implementationin LiteRT-LM serveGoogle

Gemma 4 12B: The Developer Guide·Google Developers·2026-06-03

Filed alongside

Other methods under software implementation :: inference engine.

vLLM SGLang TensorRT-LLM KTransformers Unsloth Atlas inference library Dedicated serving engines Environment variable propagation to subprocesses llama.cpp vLLM language-model-only mode vLLM or SGLang serving vLLM tensor parallelism vLLM-Ascend xLLM