Open Models Almanac
Methodsinference & servinginference kernel

single mention · inference & serving

Breakable CUDA graphs

A breakable CUDA graph captures the whole forward pass in one pass and, at each operation that cannot be captured, ends the capture, runs that operation eagerly and resumes, with no compiler involved. SGLang introduced it in February 2026 and vLLM adopted it, using it by default for several new architectures on NVIDIA GPUs, with compilation switched off.

Written for the explainer Kernels and graph compilation: where the cost model meets the GPU.

Filed under inference & serving › inference kernel