single mention · inference & serving
Write-back external KV-cache policy
Separates prefix retention from GPU residency using a write-back design.
The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See KV cache management for the methods filed beside it.
Also called write-back design.
Filed under inference & serving › KV cache management