Open Models Almanac
Methodsinference & servingKV cache management

single mention · inference & serving

Persistent per-dialogue-context KV caching

Persists a cache per dialogue context so later turns reuse prior KV and prefill only the new suffix.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See KV cache management for the methods filed beside it.

Also called Context Caching.

Filed under inference & serving › KV cache management