single mention · inference & serving
Shared-free-list cache allocation
Allocates blocks for all cache groups from one shared free list.
The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See KV cache management for the methods filed beside it.
Filed under inference & serving › KV cache management