Open Models Almanac
Methodsinference & servingKV cache management

single source · inference & serving

KDA-aware prefix-cache management

It packs fixed-size KDA states into the same paged block pool as the per-token MLA KV cache, unifying page sizes so both share one allocation, reference-counting, and eviction implementation.1

Trade-off It keeps million-token prefixes cheap to retain and reusable across requests.2

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. We therefore pack KDA states into the same paged block pool as MLA KV , unifying pages to the same byte size so that both page types share one implementation of allocation, reference counting, and eviction — k3 tech report
  2. keeping million-token prefixes cheap to retain and reusable across requests — k3 tech report

Also called joint KDA–MLA prefix cache management, KDA-aware prefix cache, cache-aware prefix cache.

Filed under inference & serving › KV cache management