Open Models Almanac
Methodsinference & servingKV cache management

single mention · inference & serving

KV-cache-aware placement of one-shot option messages

Places per-request one-shot option messages after input history so changes do not invalidate the history KV cache.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See KV cache management for the methods filed beside it.

Filed under inference & serving › KV cache management