Open Models Almanac
Methodsinference & servingKV cache management

specific method · filed under inference & serving

RadixCache

A radix cache keeps cached prefixes in a tree whose edges are runs of tokens, so a lookup follows the tree from the root and a hit can end at any token. SGLang's RadixAttention evicts least-recently-used leaves, so shared ancestors such as a system prompt survive longest.

Written for the explainer Disaggregation and the KV-cache system: scheduling, prefix reuse and tiered memory.

Also called radix cache.

sources
2
models
2
lab adopt it
1
strongest
used

Documented in

Z.ai (Zhipu AI)
Xiaomi

Evidence

2 quotes from 2 sources, one card per model, each opening with its strongest.

How sources treat it
used 1not used 1

MiMo-V2.5

Xiaominot used1 source

Unlike the commonly-used radix cache in current inference engines, our request-level prefix cache avoids re-prefilling or inter-request output cache sharing

not usedinference servingin MiMo-V2-FlashXiaomi
↑