Model techniques map
PublishersNVIDIA

NVIDIA

8 documents read from this publisher.

nemotron-3.5-lightning-30b-a3b Model by NVIDIA | NVIDIA NIM

first party release · vendor docs · 2026-09-02 · 21 techniques

Multi-Token PredictionSpeculative decodingNVFP4Mamba-2, Mixture-of-Experts, and Selective Attention HybridBest-of-N scaffoldingKnowledge distillation for synthetic dataPost-training quantizationCounterfactual data augmentationDeduplicationGitHub CrawlHeuristic filteringKeyword- and regex-based filteringPathological repetition filteringUnified data filtering pipelineBias auditsFine-tuning with demographically balanced datasetsPermissive-license filteringReproducible evaluation recipes in NeMo GymStructural checks for malformed examplesV-model iterative testing and validationVerified CUDA kernels

nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 · Hugging Face

first party release · model card · 2026-08-14 · 11 techniques

Multi-Token PredictionMulti-Token PredictionGroup Relative Policy OptimizationSupervised fine-tuningDSparkDFlashAsynchronous reinforcement learningMamba-2, Mixture-of-Experts, and Selective Attention HybridNVFP4 pre-trainingKnowledge distillation for synthetic dataUnified data filtering pipeline

nvidia/nemotron-3-ultra-550b-a55b

first party release · vendor docs · 2026-07-23 · 77 techniques

Mixture of ExpertsMulti-Token PredictionSpeculative decodingGroup Relative Policy OptimizationMulti-Teacher On-Policy DistillationFP8 KV-cache quantizationExpert ParallelismNVFP4EAGLELatentMoEAsynchronous reinforcement learningvLLMChunked prefillMamba-2Tensor ParallelismBF16NVFP4 pre-trainingPrefix cachingAttention data parallelismBest-of-N scaffoldingOpenCodeSGLangFlashInferKnowledge distillation for synthetic dataReinforcement Learning from Human FeedbackStochastic RoundingTensorRT-LLMCounterfactual data augmentationData DesignerDeduplicationGitHub CrawlHarborHeuristic filteringKeyword- and regex-based filteringMXFP8NeMo Evaluator SDKNeMo GymNVFP4 KV-cache quantizationPathological repetition filteringPipeline Parallelism

nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 · Hugging Face

first party release · model card · 2026-06-24 · 23 techniques

Mixture of ExpertsMulti-Token PredictionGroup Relative Policy OptimizationMulti-Teacher On-Policy DistillationFP8 KV-cache quantizationExpert ParallelismReinforcement LearningEAGLELatentMoEAsynchronous reinforcement learningChunked prefillMamba-2NVFP4 pre-trainingPrefix cachingFlashInferMegatron-LMData DesignerMixture-of-Experts layersNeMo Evaluator SDKNeMo GymAttention layersNemotron-CCStochastic Rounding for Mamba Cache

Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning

first party release · technical report · 2026-06-12 · 64 techniques

Mixture of ExpertsMulti-Token PredictionConfigurable reasoning effortSpeculative decodingGroup Relative Policy OptimizationMulti-Teacher On-Policy DistillationSupervised fine-tuningFP8 KV-cache quantizationExpert ParallelismNVFP4Reinforcement LearningLatentMoEChain-of-thought reasoningDiscard-all context managementGenerative Reward ModelvLLMHybrid Mamba-TransformerMamba-2Tensor ParallelismMulti-Token Prediction BoostingNVFP4 pre-trainingWarmup-Stable-DecayHugging Face TransformersHybrid Mamba-AttentionPrefill-decode disaggregationContinual pretraining for long-context extensionFour-Over-SixIcePopLogit matchingMegatron-LMOffline checkpoint mergingPeriodic cache checkpointingReinforcement Learning from Human FeedbackReinforcement Learning from Verifiable RewardsAsynchronous checkpointingBlock-scaled INT8 quantization with stochastic roundingFlashInfer NVLinkOneSidedFP8 E4M3 quantizationGranular Mixture of ExpertsLength-aware best-fit packing

NVIDIA Nemotron 3: Efficient and Open Intelligence

first party release · technical report · 2025-12-24 · 20 techniques

Multi-Token PredictionLatentMoEAsynchronous reinforcement learningMamba-2, Mixture-of-Experts, and Selective Attention HybridHybrid Mamba-TransformerBF16Multi-Token PredictionInference-time reasoning budget controlContinual pretraining for long-context extensionMXFP8GRPO with Masked Importance SamplingHigh-precision final network layersLatent-dimension reductionMulti-environment reinforcement learningNVFP4 fine-grained micro-block scalingNVFP4 TrainingOmitting RoPE in attention layersRandom Hadamard TransformsStochastic Rounding of GradientsSupervised fine-tuning at 256k sequence length

NVIDIA Nemotron AI Models

first party release · vendor docs · 2025-09-15 · 12 techniques

Hybrid Mamba-TransformerTensorRT-LLMLoRA fine-tuningBash computer-use agentConfigurable routing profiles for model selectionNeMo GuardrailsNeMo SwitchyardNemotron ParseReinforcement learning datasets with trajectories, tool calls, and preference signalsReinforcement learning with verifiable rewardsSynthetic data generation, filtering, and curationSynthetic privacy-preserving personas

NVIDIA Nemotron 3 Ultra Technical Report

technical report · 123 techniques

Mixture of ExpertsMulti-Token PredictionConfigurable reasoning effortSpeculative decodingMulti-Token PredictionGroup Relative Policy OptimizationMulti-Teacher On-Policy DistillationSupervised fine-tuningFP8Expert ParallelismNVFP4Reinforcement LearningLatentMoEDiscard-all context managementGenerative Reward ModelShared ExpertsvLLMHybrid Mamba-TransformerMamba-2Tensor ParallelismBF16Cosine decayMulti-Token Prediction BoostingNVFP4 pre-trainingWarmup-Stable-DecayHugging Face TransformersHybrid Mamba-AttentionInference-time reasoning budget controlOpenCodePrefill-decode disaggregationContinual pretrainingFour-Over-SixLogit matchingMegatron-LMOffline checkpoint mergingPeriodic cache checkpointingPost-training quantizationReinforcement Learning from Verifiable RewardsStochastic RoundingAsynchronous checkpointing