single mention · inference & serving
Multi-wave KV-cache splitting
Splits the KV cache and processes the work in multiple waves for decode attention.
The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See inference kernel for the methods filed beside it.
Also called multi-wave KV split.
Filed under inference & serving › inference kernel