Inkling
Thinking Machines LabThe chart below shows the average price customers are actually paying after prompt caching.
general family · filed under inference & serving
Caches prompts to reduce inference cost; the evidence does not specify the cache implementation.
The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See KV cache management for the methods filed beside it.
4 quotes from 4 sources, one card per model, each opening with its strongest.
The chart below shows the average price customers are actually paying after prompt caching.
The chart below shows the average price customers are actually paying after prompt caching.
The chart below shows the average price customers are actually paying after prompt caching.
average price customers are actually paying after prompt caching