general family · filed under inference & serving
CUDA Graph
A CUDA mechanism for capturing and replaying a sequence of operations; the evidence includes integration, capture, and basic caching uses.
Also called CUDA Graph integration, the step is captured in a CUDA graph, CUDA graphs and basic caching.
- sources
- 3
- models
- 3
- labs adopt it
- 3
- strongest
- used
How sources treat it
One count per evidence span, weakest treatment to strongest.
used 3
Documented in
Evidence
3 spans quoted from the sources, strongest treatment first.
including baseline implementation, autotune configuration generation, performance bottleneck diagnosis, CUDA Graph integration, persistent kernel rewriting, and host-side scheduling optimization.
usedsoftware implementationin MiniMax-M3MiniMax
the step is captured in a CUDA graph
usedunclearin Qwen3.8-Flash-NextQwen
incl. using CUDA graphs and basic caching
usedsoftware implementationin gpt-oss reference implementationsOpenAI
Filed alongside
Other methods under inference & serving :: inference kernel.
Batch-invariant deterministic kernelsCUDA Graph capture size reductionFlashAttention 3Kernel fusionMoE-side chunkingSynchronization-free static-shape MoE executionTensorRT-LLM multi-head attention backendAvoiding split-KDeepGEMM-based batch-invariant matrix multiplicationDistributed shared memory for cross-SM attention data exchangeDual-kernel batch-invariant attention decodingDynamic load balancingExp-free TopK kernelExpert-optimized Triton kernelsFA4 sheared-bias attention kernelFlashAttentionFlashAttention 4FlashKDAFP8 GEMMFused AttnRes merge and RMSNorm kernelFused latent down-projection and MoE-router GEMMFused mHC kernelsFused QSA kernelFused RoPE-attention-RoPE-cast kernel