Qwen3.8-Flash-Next
Alibaba (Qwen)the step is captured in a CUDA graph
general family · filed under inference & serving
CUDA Graph capture and replay records a fixed-shape computation as a graph and replays it, largely eliminating kernel launch overhead.1 It speeds up workloads with regular, repeated computation patterns, while sequential, variable-length execution is less amenable to graph capture.2,3
Trade-off It gives large speedups on regular fixed-shape workloads but smaller gains when execution is autoregressive and variable-length.4
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
whose runtime is dominated by kernel launch overhead, which graph replay largely eliminates— MolmoAct2 Action Reasoning Models for Real-World Deployment
its fixed-shape flow-matching steps form a regular, repeated computation pattern— MolmoAct2 Action Reasoning Models for Real-World Deployment
whose sequential dependencies and variable-length execution are less amenable to graph capture— MolmoAct2 Action Reasoning Models for Real-World Deployment
MolmoAct2-Think sees a smaller gain, since its adaptive-depth stage performs autoregressive decoding— MolmoAct2 Action Reasoning Models for Real-World Deployment
Also called CUDA Graph, CUDA Graph integration, the step is captured in a CUDA graph, CUDA Graph replay, CUDA graphs and basic caching.
6 quotes from 4 sources, one card per model, each opening with its strongest.
the step is captured in a CUDA graph
including baseline implementation, autotune configuration generation, performance bottleneck diagnosis, CUDA Graph integration, persistent kernel rewriting, and host-side scheduling optimization.
For regenerated depth tokens, we capture the fixed-shape transformer work from post-attention through the next layer’s pre-attention as CUDA Graph stages, and leave attention itself eager
We further capture the fixed-shape flow loop with CUDA Graphs, reducing Python and kernel-launch overhead during repeated action generation.
Enabling CUDA Graphs gives the larger gain
incl. using CUDA graphs and basic caching