specific method · filed under inference & serving
Persistent per-dialogue-context KV caching
Keeps a persistent cache per dialogue context so later turns can reuse prior KV and prefill only the new suffix.
Also called Context Caching.
- source
- 1
- models
- 2
- lab adopt it
- 1
- strongest
- core
How sources treat it
One count per evidence span, weakest treatment to strongest.
core 1
Documented in
Evidence
1 span quoted from the sources, strongest treatment first.
each dialogue context has a persistent key: within one policy version, later turns hit the cached KV—including generated tokens—and prefill only the new suffix.
coreinference servingin MiMo-V2.6Xiaomi
Filed alongside
Other methods under inference & serving :: KV cache management.
Chunked prefillPrefix cachingPrompt cachingSWA Bounded ReplayLanguage-model-only serving modePeriodic cache checkpointingCompressed KV cachingKDA-aware prefix-cache managementRadixCacheZero SWA caching8-bit Mamba cache quantizationAsynchronous cache offloading and restorationAutomatic cacheBlock-based KV cacheCross-layer KV-cache reuseCustomized heterogeneous KV-cache layoutDisable prefix caching for benchmarkingDynamic allocation of fixed-size state-cache poolsEncoder SWA bounded replayExact SWA KV reconstruction via full multi-layer replayFine-grained prefix hashingHierarchical KV cachingInference-side KV-cache reset on weight synchronizationKDA with prefill cache