specific method · filed under inference & serving
Chunked prefill
Processes prefill in chunks; the evidence also mentions retaining chunked encoding for temporal audio and vision outputs.
Also called --chunked-prefill-size 16384, chunked prefilling, chunked-prefilling mechanism.
- sources
- 6
- models
- 4
- labs adopt it
- 3
- strongest
- default
How sources treat it
One count per evidence span, weakest treatment to strongest.
Documented in
Further reading
Picked by hand, not extracted: where to read more, not evidence for anything on this page.
Evidence
6 spans quoted from the sources, strongest treatment first.
All deployment snippets below default to port 8000, with chunked prefill, NVFP4 KV caching, and MTP (5 speculative tokens) enabled
enable-chunked-prefill
--enable-chunked-prefill
we retain the chunked-prefilling mechanism as implemented in Qwen3-Omni and Qwen2.5-Omni, whose audio and vision encoders are capable of outputting chunks along the temporal dimension.
--chunked-prefill-size 16384
--chunked-prefill-size 16384
Filed alongside
Other methods under inference & serving :: KV cache management.