Prefix caching reuses computation from those shared inputs.
specific method · filed under inference & serving
Prefix caching
Prefix caching keeps the KV cache of finished requests and reuses it when a new request starts with the same tokens, so the server computes only the new part of the prompt. The new tokens still attend over the cached prefix, so a hit costs their weight matmuls plus that attention, and a single changed token early in the prompt turns the rest into a miss.
Written for the explainer Disaggregation and the KV-cache system: scheduling, prefix reuse and tiered memory.
Also called prefix cache, enable-prefix-caching.
Documented in
Evidence
10 quotes from 10 sources, one card per model, each opening with its strongest.
Mistral-Large-4
Mistral AIPrefix caching (on by default) for agentic workloads
1 more from this model
--enable-prefix-caching \
enable-prefix-caching
1 more from this model
--enable-prefix-caching
GLM-5.2
Z.ai (Zhipu AI)Prefix Cache (extending KV storage to system RAM)
Qwen3.6-35B-A3B
Alibaba (Qwen)--enable-prefix-caching
Step-5-Preview
StepFun`--enable-prefix-caching`
1 more from this model
--enable-prefix-caching
Qwen3.5-397B-A17B
Alibaba (Qwen)enable MTP-1 speculative decoding and disable prefix caching.
Further reading
Picked by hand, not extracted: where to read more, not evidence for anything on this page.
- SGLang: Efficient Execution of Structured Language Model Programs (Zheng et al., 2023) paper arxiv.orgintroduces RadixAttention prefix caching
- Automatic Prefix Caching - vLLM Documentation docs docs.vllm.ai