Open Models Almanac
Methodsinference & servinginference kernel

single mention · inference & serving

Avoiding split-K for batch invariance

An approach that avoids split-K matrix multiplication in most scenarios to preserve batch invariance, potentially at a performance cost.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See inference kernel for the methods filed beside it.

Also called we abandon split-k in most scenarios.

Filed under inference & serving › inference kernel