Open Models Almanac
Methodsinference & servingKV cache management

implementation detail · filed under inference & serving

Internal prefill checkpointing

Creates an internal cache checkpoint during prompt processing, with one described implementation extracting Mamba state at a logical cache boundary within a single prompt forward.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See KV cache management for the methods filed beside it.

Also called internal checkpointing.

sources
2
models
2
lab adopt it
1
strongest
used

Documented in

Evidence

2 quotes from 2 sources, one card per model, each opening with its strongest.

How sources treat it
used 2

The adapted path arranges for a logical chunk to end at the desired cache boundary, extracts the state and convolution window there, and writes that checkpoint into the paged cache while the full prompt remains in one model forward.

usedinference servingin Nemotron 3.5 SuperNVIDIA
1 more from this model

internal checkpointing avoids extra model passes during prompt processing.

usedinference servingin Nemotron 3.5 Super VLNVIDIA
↑