the Nemotron 3 family of models use a Mixture-of-Experts hybrid Mamba-Transformer architecture that pushes the accuracy-to-inference-throughput frontier
3 more from this model
both models combine large-scale distillation, long-context training, and native speculative decoding
powered by hybrid Mamba‑Transformer MoE with 1M-token context
Nemotron 3 models predominantly interleave MoE layers with cheaper Mamba-2 layers