single mention · inference & serving
W4A16 quantization of Mamba output-projection weights
Mamba output-projection weights are quantized to 4-bit while activations remain at 16-bit precision.
The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See inference quantization for the methods filed beside it.
Also called Mamba output projection→ W4A16.
Filed under inference & serving › inference quantization