GLM-5.2
Z.ai (Zhipu AI)Context Management: Features like RadixCache (prefix sharing) and Prefix Cache
specific method · filed under inference & serving
A radix cache keeps cached prefixes in a tree whose edges are runs of tokens, so a lookup follows the tree from the root and a hit can end at any token. SGLang's RadixAttention evicts least-recently-used leaves, so shared ancestors such as a system prompt survive longest.
Written for the explainer Disaggregation and the KV-cache system: scheduling, prefix reuse and tiered memory.
Also called radix cache.
2 quotes from 2 sources, one card per model, each opening with its strongest.
Context Management: Features like RadixCache (prefix sharing) and Prefix Cache
Unlike the commonly-used radix cache in current inference engines, our request-level prefix cache avoids re-prefilling or inter-request output cache sharing