Open Models Almanac
Methodsinference & servingKV cache management

general family · filed under inference & serving

Prompt caching

Caches prompts to reduce inference cost; the evidence does not specify the cache implementation.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See KV cache management for the methods filed beside it.

sources
4
models
4
labs adopt it
3
strongest
used

Documented in

Google DeepMind
Thinking Machines Lab

Evidence

4 quotes from 4 sources, one card per model, each opening with its strongest.

How sources treat it
used 4
↑