general family · filed under inference & serving
Kernel fusion
Kernel fusion merges a chain of GPU operations, such as a normalization and the rounding to FP8 that follows it, into one kernel, so the intermediate result stays in registers or shared memory instead of being written to and read back from HBM. It saves memory traffic and launches, and each fused kernel is written for one chain of operations, one layout and one format.
Written for the explainer Kernels and graph compilation: where the cost model meets the GPU.
Also called inference kernel fusion, systematic kernel fusing, operator fusion.
Documented in
Mistral AI
Tencent (Hunyuan)
Evidence
2 quotes from 2 sources, one card per model, each opening with its strongest.
How sources treat it
used 2
Hy4-preview
Tencent (Hunyuan)optimization on areas such as operator fusion and communication optimization
usedunclearin Tencent Hy4 previewTencent