single mention · inference & serving
On-disk KV-cache storage for shared-prefix reuse
Stores KV cache on disk to avoid repeated prefilling for requests with shared prefixes.
The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See KV cache management for the methods filed beside it.
Also called on-disk KV cache storage mechanism.
Filed under inference & serving › KV cache management