Open Models Almanac
Methodsinference & servingKV cache management

specific method · filed under inference & serving

Hybrid-state caching

Caches hybrid model state for serving; the evidence does not specify a more particular mechanism.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See KV cache management for the methods filed beside it.

sources
2
models
2
lab adopt it
1
strongest
core

Documented in

Evidence

2 quotes from 2 sources, one card per model, each opening with its strongest.

How sources treat it
used 1core 1

Hybrid-state caching, prefix reuse, Mamba checkpointing, GPU-kernel selection, and speculative decoding are jointly tuned

coreinference servingin Nemotron 3.5NVIDIA
1 more from this model

We additionally optimize hybrid-state caching and GPU kernels for efficient long-context and agentic serving

usedinference servingin Nemotron 3.5 SuperNVIDIA
↑