The fix is MoE-side chunking, splitting the token batch within the MoE kernel itself
usedsoftware implementationin vLLMNVIDIA
1 more from this model
The fix is MoE-side chunking, splitting the token batch within the MoE kernel itself
usedinference servingin Nemotron 3 UltraNVIDIA