NVIDIA
NVIDIA-Nemotron-3.5-Super-VL-120B-A12B
What NVIDIA-Nemotron-3.5-Super-VL-120B-A12B’s own documents say it is built from — every method with the sentence that describes it.
- Parameters
- 120B / 12B active
- Context
- 1M
- Attention
- Mamba hybridhybrid Mamba-Transformer (nemotron_h_omni), RADIO vision encoder
- Experts
- LatentMoE, 512 routed + 1 shared, top-22
- Input
- text, image, video
- Licence
- OpenMDW-1.1
- Released
- 2026-10-15
Against the consensus recipe
Its documents state 8 of the 18 methods the field agrees on.
Data curation
- Benchmark decontaminationused
- Synthetic data generationnot in its documents
- Corpus deduplicationused
Training objective
- Multi-Token Predictionnot in its documents
- Knowledge insulationnot in its documents
- Quantization-aware distillationused
Model architecture
- Mixture of expertscore
- Multi-Token Predictioncore
- Grouped-query attentionnot in its documents
Optimization
- Muonnot in its documents
- Expert parallelismnot in its documents
- Quantization-Aware Trainingnot in its documents
Post-training
- Supervised fine-tuningused
- Asynchronous reinforcement learningnot in its documents
- Multi-Teacher On-Policy Distillationused
Inference & serving
- Thinking modeoptional
- Multi-Token Predictionnot in its documents
- FP8not in its documents
Checked in its code
Its code shows 5 of the 19 architecture features checked — 3 of them are not stated in its documents.
Sources say what a lab chose to describe; the modeling code says what the checkpoint runs. Each feature below was put to a code verifier, and only the config values and source lines it cited that were found in the files are shown, linked to the line.
Read from Hugging Face Transformers (modeling_nemotron_h_omni.py, configuration_nemotron_h_omni.py) and config.json at revision fb0ee0bc6b, by anthropic/claude-sonnet-5 on 2026-11-07.
| Feature | Code | Its documents | Verdict |
|---|---|---|---|
| Grouped-query attention code only | in its code | not stated | The text_config sets 32 attention heads with only 2 key-value heads, indicating grouped-query attention in the attention layers, though… show evidenceThe text_config sets 32 attention heads with only 2 key-value heads, indicating grouped-query attention in the attention layers, though the nemotron_h attention implementation file itself was not provided. llm_config.num_attention_heads = 32llm_config.num_key_value_heads = 2 |
| Multi-head latent attention | code unclear | not stated | No kv_lora_rank/q_lora_rank keys appear in the text_config and the nemotron_h attention code is not included, so MLA cannot be confirmed… show the whole verdictNo kv_lora_rank/q_lora_rank keys appear in the text_config and the nemotron_h attention code is not included, so MLA cannot be confirmed or ruled out. |
| Sliding-window attention | not in its code | not stated | sliding_window is explicitly null in the text_config, so no sliding-window restriction is applied even if supported elsewhere. show evidencellm_config.sliding_window = null |
| Interleaved sliding-window and global attention | not in its code | not stated | Since sliding_window is null, there is no interleaving of sliding and full attention; layer typing only alternates mamba/moe/attention… show evidenceSince sliding_window is null, there is no interleaving of sliding and full attention; layer typing only alternates mamba/moe/attention, not windowed vs global attention. llm_config.sliding_window = null |
| Indexer-selected sparse attention (DeepSeek Sparse Attention) | not in its code | not stated | No index_topk, index_n_heads, or index_head_dim keys appear anywhere in the config, and no indexer code is present in the given files. |
| Linear-attention or state-space layers alongside full attention | code unclear | not stated | layers_block_type interleaves 'mamba' (a linear-complexity SSM mixer) with 'moe' feed-forward blocks and occasional 'attention' softmax… show the whole verdictlayers_block_type interleaves 'mamba' (a linear-complexity SSM mixer) with 'moe' feed-forward blocks and occasional 'attention' softmax layers in the text_config. |
| Gated DeltaNet layers | code unclear | not stated | The config has mamba-specific keys (mamba_num_heads, conv_kernel, ssm_state_size) rather than linear_num_key_heads/linear_conv_kernel_dim… show the whole verdictThe config has mamba-specific keys (mamba_num_heads, conv_kernel, ssm_state_size) rather than linear_num_key_heads/linear_conv_kernel_dim used by Gated DeltaNet, and the mixer implementation file is not provided, so it is unclear whether the mamba layers use the gated delta rule versus standard Mamba-2 SSD. |
| Mamba-2 layers code only | in its code | not stated | The text_config declares Mamba-2-style SSM hyperparameters (ssm_state_size, mamba_num_heads, mamba_head_dim, chunk_size) used by the… show evidenceThe text_config declares Mamba-2-style SSM hyperparameters (ssm_state_size, mamba_num_heads, mamba_head_dim, chunk_size) used by the 'mamba' layers in layers_block_type, matching NemotronH's Mamba-2 mixer design. llm_config.ssm_state_size = 128llm_config.mamba_num_heads = 128llm_config.mamba_head_dim = 64llm_config.chunk_size = 128 |
| Learnable attention sink | code unclear | not stated | No attention-sink related keys or code are present in the given config/files to confirm or rule out a learned sink logit in the attention… show the whole verdictNo attention-sink related keys or code are present in the given config/files to confirm or rule out a learned sink logit in the attention layers. |
| Gated attention output | code unclear | not stated | No attn_output_gate/use_output_gate keys appear and the nemotron_h attention implementation is not included, so an input-dependent sigmoid… show the whole verdictNo attn_output_gate/use_output_gate keys appear and the nemotron_h attention implementation is not included, so an input-dependent sigmoid output gate cannot be confirmed. |
| QK normalization | code unclear | not stated | No use_qk_norm/qk_layernorm keys appear in the config and the attention implementation file is not provided, so QK normalization cannot be… show the whole verdictNo use_qk_norm/qk_layernorm keys appear in the config and the attention implementation file is not provided, so QK normalization cannot be confirmed. |
| Partial RoPE | code unclear | not stated | No partial_rotary_factor, qk_rope_head_dim, or rotary_dim keys appear in the text_config, and the attention/rope code is not included, so… show the whole verdictNo partial_rotary_factor, qk_rope_head_dim, or rotary_dim keys appear in the text_config, and the attention/rope code is not included, so partial RoPE cannot be determined. |
| YaRN RoPE scaling | not in its code | not stated | No rope_scaling or rope_parameters key is present anywhere in the config, so no YaRN scaling is configured. |
| Layers without positional encoding (NoPE) | code unclear | not stated | No no_rope_layers or nope_layer_interval keys appear in the config, and the attention implementation deciding RoPE application per layer… show the whole verdictNo no_rope_layers or nope_layer_interval keys appear in the config, and the attention implementation deciding RoPE application per layer is not provided. |
| Mixture of experts | in its code | adopted | The text_config defines 512 routed experts with 22 selected per token, used by the 'moe' blocks in layers_block_type. |
| Shared expert code only | in its code | not stated | The text_config declares 1 shared expert with a nonzero intermediate size, applied alongside the routed experts in MoE layers. |
| Auxiliary-loss-free load balancing (selection bias) | code unclear | not stated | No topk_method or e_score_correction_bias key is present in the config, and the MoE router code is not included, so bias-based load… show the whole verdictNo topk_method or e_score_correction_bias key is present in the config, and the MoE router code is not included, so bias-based load balancing cannot be confirmed. |
| Multi-token prediction layers | in its code | adopted | The text_config declares num_nextn_predict_layers=1 and an mtp_layers_block_type, and the omni modeling file conditionally builds an extra… show evidenceThe text_config declares num_nextn_predict_layers=1 and an mtp_layers_block_type, and the omni modeling file conditionally builds an extra vision norm specifically for checkpoints whose language model carries MTP layers, confirming MTP modules are active for this config. llm_config.num_nextn_predict_layers = 1 |
| SwiGLU feed-forward | not in its code | not stated | The text_config sets mlp_hidden_act to 'relu2' rather than a SiLU-gated activation, so the feed-forward/expert MLPs are not SwiGLU. show evidencellm_config.mlp_hidden_act = "relu2" |
filed at the root 1
“The model builds on a strong foundation and uses a hybrid architecture, speculative decoding, context reuse, and mixed precision”
token mixer 4
“each of the five layers of the drafter body wraps the attention and MLP sublayers in a grouped dynamic convolution”
softmax attention 1
sliding window attention 1
“We therefore serve it with sliding-window attention over the most recent 8,192 positions in every drafter layer”
linear attention & state space 1
Mamba 1
“Hybrid-state caching, prefix reuse, Mamba checkpointing, GPU-kernel selection, and speculative decoding are jointly tuned”
hybrid layer stacking 1
“both models combine large-scale distillation, long-context training, and native speculative decoding”
channel mixer 2
mixture of experts 2
“Its 120B-parameter mixture-of-experts (MoE) architecture activates 12B parameters per token”
expert load balancing 1
“Router-bias updates are explicitly disabled.”
prediction head 8
“Multi-token prediction (MTP) gives Nemotron 3.5 Super VL an internal drafter that proposes several tokens for the main model to verify in…”
“the backbone remains frozen and only the MTP head parameters are updated.”
“A shared MTP head predicts future tokens recursively for speculative decoding.”
“adopting the recursively shared MTP formulation”
“input hidden states are sampled from the sequence of states produced by earlier MTP steps 1,...,k − 1, rather than relying exclusively on…”
“We first fuse the latent MoE projections into the experts’ up- and down-projection matrices, then prune the hidden size from 4096 to 2688”
“conduct a dedicated MTP warmup phase for 100 B tokens, freezing the backbone parameters and optimizing only the MTP head”
“We apply a dedicated MTP-only training recipe at two points”
multimodal architecture 3
“The vision encoder and connector turn visual inputs into representations processed by the language backbone.”
“The vision encoder and connector turn visual inputs into representations processed by the language backbone.”
“the video configuration samples up to 256 frames at 2 FPS with a pruning rate of 0.5.”
filed at the root 1
“We replace it with a calibration term.”
multi-token prediction objective 3
“A dedicated MTP-boosting training phase further aligned the drafter with the final model.”
“Whereas Nemotron 3 Ultra uses a temperature-scaled forward-KL objective, we now use the LK loss”
“Both models begin with a dedicated 100-billion-token MTP warm-up”
distillation objective 8
“The student minimizes the reverse KL𝐷KL(𝜋𝑆(·|𝑐𝑡)‖𝜋𝑇 (·|𝑐𝑡))at every position of its own responses”
“The released checkpoint uses the predefined aggressive PTQ recipe followed by SFT-based QAD at 200 iterations.”
“with Ultra 3 Base as the teacher and Skew Reverse KL divergence as the distillation loss function”
“these runs use a knowledge-distillation loss with the frozen backbone as the teacher.”
“we combined the KL loss with cross-entropy loss, assigning them weights of 0.9 and 0.1, respectively”
“We used standard forward KL divergence loss along with an extra sink token to capture the residual probability mass outside top-128.”
“We depart from this recipe and, as in MTP boosting (Section 3.7), instead distill the full next-token distribution pM of the target model M”
“We use KL divergence as our objective.”
auxiliary loss 2
“We disable the main hard-target cross-entropy loss, MTP cross-entropy loss, and MoE load-balancing auxiliary loss and don’t use…”
“We mask out prompt, conversation-history, tool-output, and prefilled response-prefix tokens.”
optimization 15
optimizer 1
“Optimization uses distributed AdamW with β1 = 0.9, β2 = 0.95, ϵ = 10−8, weight decay 0.1, and gradient clipping at a global norm of 1.0.”
learning-rate schedule 1
“We reset the learning rate, warmed it up linearly to a maximum of1× 10−5, about twice the final pretraining learning rate, and then held…”
training precision 2
“It uses NVFP4 quantization to reduce memory requirements”
“two optimizations designed to mitigate train–inference mismatch: router replay (Ma et al., 2025) and using FP32 for LM head (Chen et al.…”
quantization-aware training 4
“such as randomness generation, stochastic rounding, or scaling calculations.”
“Activation scales are learned from a calibration set of image–text pairs, rather than text alone.”
“All other parameters remain frozen, including token embeddings, unquantized language-model parameters, normalization parameters, the…”
“The released checkpoint uses the predefined aggressive PTQ recipe followed by SFT-based QAD at 200 iterations.”
training stability 2
“gradient clipping at a global norm of 1.0.”
“augmented with two optimizations designed to mitigate train–inference mismatch: router replay (Ma et al., 2025) and using FP32 for LM head”
training parallelism 3
“We used 16-way context parallelism, 8-way tensor parallelism, and 64-way expert parallelism to train on GB300 GPUs.”
“We used 8-way context parallelism, 8-way tensor parallelism, 8-way expert parallelism, and 2-way pipeline parallelism to train on GB200…”
“Pipeline parallelism left only one GPU active at a time, so we chose DP× EP.”
2 more mentioned or evaluated
training precision “Runaway moves by ≈ 0; output verbosity falls on 5 of 5 focused benchmarks”
training precision “No recovery of termination is captured.”
data filtering 15
“We therefore performed checks of post-training data against 30 benchmarks (see the complete list in Appendix C.2) and removed samples…”
“As we did not have reliable multilingual model-based quality classifiers available, we applied just heuristic filtering instead”
“We collect raw source code and subsequently remove any having a license which does not exist in our permissive-license set.”
“we apply automated quality checks to filter ambiguous or visually unsupported questions, incomplete reasoning traces, hallucinated labels…”
“Every source is decontaminated against SWE-bench and Terminal-Bench and has secrets removed.”
“retaining samples that are nearly impossible to solve correctly without using the available tools.”
“we removed reasoning responses shorter than 5,000 tokens to focus the training mixture on examples containing sufficiently developed…”
“Verbatim overlap measures how many n-grams of the question’s text appear in the sample.”
“The sample is reduced to its problem statement, the part into which a benchmark question could leak. Text templates that recur across its…”
“We retain normally completed trajectories whose answers match the references and whose required tools execute successfully.”
“per question: top 20 by combined score+top 2 per signal”
“We audit reference-answer formats and compatibility with the reward functions, and remove exact question–answer duplicates from the…”
“A sample is removed if the judge finds it severely contaminated with respect to any benchmark question.”
“Synthetic trajectories are filtered for structural correctness, repetition, and task validity.”
“We keep translations scoring at least 75.”
deduplication 2
“The data was subsequently deduplicated and filtered in various ways used in the Nemotron-CC data pipeline.”
“remove exact question–answer duplicates from the second-stage mixture.”
synthetic data 17
“We use Kimi K2.6 (Kimi Team, 2025) and K3 (Kimi Team, 2026) to generate agentic datasets with a focus on image tool operations such as…”
“For the ten additional lower-resource languages, we use Bodhan-MT for translation and Gemma-4-31B-it for quality assessment, with…”
“we synthetically generate long-context translation data from both the PG19 (Rae et al., 2019) and FinePDFs-Edu (Kydlíček et al., 2025)…”
“we improved scheduling so translation and quality checks can run concurrently.”
“The agentic track targets gaps found in a coverage audit against the Vals.ai category taxonomy by making the relationship among source…”
“The document-grounded track samples windowed contexts from SEC 10-K, 10-Q, and 8-K filings of S&P 500 companies, generates analyst-style…”
“Synthetic trajectories are filtered for structural correctness, repetition, and task validity.”
“We implement the four-stage pipeline for the twelve priority languages”
“We use overlapping windows only for longer Santali documents”
“Lower-scoring translations are rewritten using the source text and identified errors, then checked again and kept only if they pass.”
“The model then checks each translation against its source and assigns a quality score from 0 to 100.”
“organizing the reasoning traces into an explicit sequence of concise steps, with each step grounded in relevant visual evidence, helps…”
“Diverse training data generated with NeMo Data Designer helped the model better understand complex charts.”
“we synthetically generated graph-structured data to teach the model to trace nodes and edges through reasoning, rather than relying solely…”
“We address these gaps by translating curated English documents into Indic languages.”
“long-translation performance was improved by also including synthetic transduction tasks (Willette et al., 2026) that reinforce position…”
“translate each document in full, without splitting”
data mixture & curriculum 6
“we train a vision tool-calling teacher on a curated mixture of internal data.”
“are routed to the SFT or RL pool according to their measured difficulty.”
“10% (10B tokens) allocated to selected synthetic text from our corpus and natural Indic text.”
“Most of it is natural data kept in its original format, without chat templates.”
“We replace the broad RLVR mixture with a reasoning-focused curriculum of 25,600 examples, with each 256-prompt batch containing 90%…”
“stage 1 is weighted toward the domains that improve quickly and tend to regress later, and stage 2 toward the domains that improve more…”
sequence packing 1
“Training uses packed sequences of up to 524,288 tokens.”
2 more mentioned or evaluated
data mixture & curriculum “we ran separate Lightning 3.5 research experiments with 10%, 20%, and 30% Indic data under a fixed 200B-token training budget.”
data mixture & curriculum “a mixture of 75% SFT data with 25% teacher-generated RL rollouts”
filed at the root 1
“We further train specialized teachers for video reasoning and tracking, vision tool use, and alignment”
supervised fine-tuning 10
“A two-stage SFT pipeline trains the language and vision components”
“Super’s post-training pipeline combines supervised fine-tuning”
“we do a small second stage of SFT, where we change the blend composition to boost some capabilities the first stage did not learn…”
“The agent traces are the only SFT-style data, so we keep them to a small share of the blend and mask tool responses from the loss…”
“Dozens of organizations across industries are post-training Super VL on their own data, tools, policies and workflows”
“the reasoning traces of a small fraction of SFT data are truncated to random reasoning budgets while the responses remain the same and…”
“Nemotron 3.5 Lightning is trained with single SFT stage with the text only SFT data.”
“we reduced checkpointing interval to 100 iterations, used the 100% SFT blend only, and evaluated QAD checkpoints at 200, 300, and 400…”
“Developers can use NVIDIA NeMo AutoModel for supervised fine-tuning (SFT) and low-rank adaptation (LoRA) to teach domain-specific behavior.”
“Developers can post-train Nemotron 3.5 Super VL with NVIDIA NeMo using their own data, tools, policies, and workflows.”
reinforcement learning algorithm 7
“then post-trained with roughly 3.8 million samples and reinforcement learning across dozens of environments.”
“We post-train Nemotron 3.5 Super through a comprehensive pipeline spanning Supervised Fine Tuning (SFT), Reinforcement Learning from…”
“We initialize from an existing multimodal RLVR checkpoint and train using asynchronous GRPO”
“We mix this RLHF training with the abstention training strategy from Nemotron 3 Ultra”
“tokens are then importance-weighted by¯𝜋𝑆/¯𝜋𝑆−1 and those with weights outside[0.2, 5]are masked”
“continue training with full end to end GRPO on agentic tasks.”
“length-based adjustments are applied on the RL rewards for them.”
reward modelling 8
“Reinforcement learning with verifiable rewards, reinforcement learning from human feedback, and specialized teacher training”
“First, we train a new variant of the Nemotron 3 Ultra GenRM that is trained with a reasoning length penalty to be more token efficient.”
“as well as an auxiliary tool-use reward to penalize duplicate and malformed tool calls.”
“Video-QA uses a binary reward based on multiple-choice answer correctness.”
“the verifier assigns a continuous reward based on localization quality, object-visibility accuracy, and the validity and completeness of…”
“Second, we apply a soft length preference only to correct responses.”
“Both sources use deterministic, verifiable task rewards rather than an LLM judge.”
“a response without exactly one properly matched <think>...</think> block is considered malformed: its task reward is set to zero and its…”
preference optimization 2
“We train this teacher primarily using RLHF.”
“we instead compare all policy responses to a fixed response from a strong open-source model, Gemma 4 31B-it.”
policy distillation 8
“using the token-local update”
“After MOPD, truncation decreases to 0.3% with 13,669 output tokens per rollout, where only one runaway loop failure is captured.”
“then distilled each model for 1 trillion tokens”
“followed by 1T tokens of distillation from Nemotron 3 Ultra Base.”
“environments no specialist owns are routed to a self-anchor, a frozen copy of the student’s starting checkpoint”
“consolidating capabilities from reasoning, agentic, alignment, video, and vision-tool-use teachers”
“the distillation target is smoothed with the proximal-teacher construction of trust-region policy distillation”
“MOPD for Nemotron 3.5 Super is run in two stages on 128 GB200 nodes.”
rollout & RL infrastructure 4
“We generate on-policy rollouts using the corresponding backbone checkpoint.”
“For each task, N independent instances of the same post-trained agent attempt the task, producing a pool of candidate trajectories. These…”
“responses may come from the previous snapshot¯𝜋𝑆−1, i.e. one step stale”
“For the RL rollouts we sample 16 trajectories per source record with vLLM.”
agentic post-training 2
“The environment provides the same tools—SEC filing search, document parsing, evidence retrieval, and final source-supported…”
“opting to train the end to end agentic teacher on sandbox, fully executable agentic Terminal, SWE, and Search tasks.”
mid-training & continual pretraining 6
“In the LC-Phase, we performed continued pretraining (CPT) at 1M-token context length to equip the base model with long-context ability.”
“Mid-training sits between the two. It trains the base model at long context on a diverse mixture of natural software-engineering data with…”
“combines synthetic text generated from English documents with natural Indic corpora to strengthen Indian-language capabilities”
“To recover long-context capabilities affected by 256K mid-training, we perform an additional 9 billion token recovery stage”
“A 65.4-billion-token stage at 256K context uses repositories, issue–pull-request pairs, commit histories, build and CI logs, and agent…”
“Continued pretraining with high-quality natural and synthetic data improved Indic-language capabilities”
1 more mentioned or evaluated
supervised fine-tuning “This SFT is an experimental agentic blend of about 33B tokens (2K steps), not the one used in Section 3”
filed at the root 1
“Rather than impose one serving configuration everywhere, we tune quantization and speculative decoding for each platform.”
decoding strategy 10
“The model builds on a strong foundation and uses a hybrid architecture, speculative decoding, context reuse, and mixed precision”
“LiLiCorr retains the K most probable tokens of the drafter body’s marginal as candidates”
“parallel drafters propose all draft tokens in a single forward pass whose latency is nearly independent of the draft length.”
“LiLiCorr instead uses a lightweight, single-pass correlation head to score compatibility across slots.”
“It keeps the drafter body parallel and adds a lightweight sequential head that biases the logits at each slot using a low-rank transition…”
“DFlash and DSpark for Nemotron 3.5 Lightning”
“DFlash and DSpark for Nemotron 3.5 Lightning”
“LiLiCorr improves parallel speculative decoding by jointly evaluating token candidates across an entire draft”
“In vLLM, num_speculative_tokens controls draft length; in SGLang, the MTP draft length uses --speculative-num-steps to config.”
“Under probabilistic draft sampling, each slot is instead drawn from the head’s distribution over its candidates at the request’s…”
reasoning control 3
“Reasoning is on by default and can be turned off or given a token budget.”
“Super supports reasoning-off, regular, and medium-effort modes”
“The regular and medium-effort reasoning modes have the option to be used in conjunction with inference-time budget control.”
KV cache management 8
“Prefix caching reuses computation from those shared inputs.”
“Hybrid-state caching, prefix reuse, Mamba checkpointing, GPU-kernel selection, and speculative decoding are jointly tuned”
“The model’s serving optimizations preserve both attention caches and Mamba recurrent states.”
“Hybrid-state caching, prefix reuse, Mamba checkpointing, GPU-kernel selection, and speculative decoding are jointly tuned”
“The adapted path arranges for a logical chunk to end at the desired cache boundary, extracts the state and convolution window there, and…”
“Additional Mamba checkpoints reduce the replay needed to recover inputs for speculative decoding”
“We have since implemented a "checkpointing replay" method.”
“NVIDIA Dynamo supports distributed inference optimizations to increase throughput and reduce latency, including KV-cache-aware routing, KV…”
inference quantization 24
“MTP KV precision therefore follows the backbone.”
“The model builds on a strong foundation and uses a hybrid architecture, speculative decoding, context reuse, and mixed precision”
“using a native 2-bit quantization to fit in 24GB of memory”
“thus a single FP8 lm_head serves both target verification and MTP drafting”
“We apply post-training quantization (PTQ) and quantization-aware distillation (QAD) with ModelOpt on Lightning 3.5 and Super 3.5 VL”
“Hence we use the MSE calibration for the PTQ and AutoQuant recipes.”
“The released checkpoint uses the aggressive recipe, with NVFP4 MoE and shared-expert GEMMs and FP8 attention projections and Mamba linear…”
“The MTP attention weights remain BF16, with only the K/V BMM quantizers at FP8.”
“Precision is selected separately for sensitive components.”
“To maintain token generation verbosity, the lm_head should stay at FP8 weights or above”
“the mixed-precision checkpoints of §4.3, produced by a gradient-based per-layer search”
“We develop NVFP4 post-training quantization and mixed-precision recipes across weights, caches, and the MTP drafter”
“We therefore quantize the MTP experts using NVFP4 W4A4.”
“quantizing LiLiCorr’s linear layers to NVFP4 while keeping the activations at BF16, resulted in a 11% throughput improvement on DGX Spark”
“To address this issue, we preferred parameter alignment as described in §4.4.”
“This study recommends FP8 precision on 𝑞/𝑘/𝑣 projection at per-block scale granularity.”
“Inference uses W4A4 on Blackwell and W4A16 on non-Blackwell platforms”
“We use the SFT data employed during post-training as the calibration corpus and calibrate the model using 1,024 samples.”
“complemented by quantization-aware distillation, achieving 99.62% median accuracy recovery relative to BF16”
“The released checkpoints were produced by quantizing the boosted MTP separately and splicing it into the selected quantized backbone”
“AutoQuant estimates a sensitivity Sᵢ(q) from the quantization-induced output error weighted by the squared task-loss gradient.”
“we applied weight-only PTQ to improve the drafting efficiency of a DSpark module for Nemotron Lightning 3.5 and LiLiCorr module for…”
“The NVFP4 recipe uses 4-bit weights and activations on the NVIDIA Blackwell architecture.”
“This allows the target lm_head to remain in BF16 while using a lower-precision lm_head for drafting.”
serving parallelism 1
“NVIDIA Dynamo supports distributed inference optimizations to increase throughput and reduce latency, including KV-cache-aware routing, KV…”
inference scheduling 4
“A KV-cache-aware router sends each request to an instance.”
“Session affinity sends all turns of one session to the same worker, so later turns find their prefix in its cache.”
“--async-scheduling”
“we can recover by routing to another healthy provider, if your request filters allow it”
inference kernel 4
“Hybrid-state caching, prefix reuse, Mamba checkpointing, GPU-kernel selection, and speculative decoding are jointly tuned”
“optimize hybrid-state caching and GPU kernels for efficient long-context and agentic serving”
“The multi-wave KV split then splits the KV cache further and runs the work in several waves.”
“The global linear backend remains the default, while a kernel configuration can override it for a specific quantization scheme”
context management 1
“The model builds on a strong foundation and uses a hybrid architecture, speculative decoding, context reuse, and mixed precision”
agentic scaffolding 1
“We further improve the final model’s agentic performance through parallel agents followed by self-verification.”
19 more mentioned or evaluated
reasoning control “we explore test-time scaling through parallel agent execution and self-verification, improving Terminal-Bench 2.1 from 62.7% to 71.9%”
inference quantization “aggregation benchmarks can identify important candidates in FP8, where even per-tensor FP8 precision can be acceptable over 100 tasks at…”
inference quantization “Expert activations → BF16. Runaway moves by ≈ 0”
inference quantization “NVFP4 stores 4-bit E2M1 values with one FP8 E4M3 scale per 16-element block”
inference quantization “quantizing its weights to FP4 inflates generation length.”
inference quantization “in each case both the key and value caches across attention layers are quantized in FP8 or NVFP4, respectively.”
inference quantization “Empirically, promoting MoE expert precision and fine-tuning on RL rollouts can mitigate this effect”
KV cache management “KV-aware routing with session affinity and KV cache offload serves up to 8x as many users”
inference quantization “with a linear E2M1 residual the KL falls to 0.00117 against per-tensor FP8’s 0.00134”
inference quantization “Substituting MXFP8 for per-tensor FP8 on 𝑞/𝑘/𝑣 changes only the scale granularity with one E8M0 exponent per 32 elements instead of one…”
inference quantization “Routed/shared experts NVFP4→ FP8. In tau3-banking, runaway dropped from 28.2% → 17.9%”
inference quantization “Further quantizing the LM head improved throughput by another 5%.”
inference quantization “MSE 4-over-6: MSE-based scale selection restricted to a narrower search space.”
inference quantization “Rotation or higher precision can mitigate this effect in activation”
inference quantization “AutoQuant was explored as an alternative sensitivity-driven mixed-precision approach.”
inference quantization “we emulated two-pass FP4 weights, where a second channel corrects the residual left below the amax/24 flush line.”
inference quantization “output verbosity increased by 18.5% at 2.4𝜎, while the runaway rate and knowledge abstention were both unaffected.”
inference quantization “Applying rotation or higher activation precision can mitigate the flush.”
inference quantization “Applying rotation or higher activation precision can mitigate the flush.”
Not used 3
decoding strategy “The Lightning 3.5 release used the lightweight Markov correction but omitted this confidence head.”
inference quantization “Per-layer mixed FP8-key/NVFP4-value (or vice-versa) configurations are not evaluated in this comparison.”
inference quantization “This tensor analysis stops us from taking the Mamba output projection to W4A4.”
filed at the root 1
“NVIDIA NIM packages the model as a production-ready microservice with performance-optimized, validated configurations”
training framework 2
“ModelScope enables developers to fine-tune the model through its ms-swift framework for specialized multimodal applications.”
“along with the reinforcement learning environments in NeMo Gym.”
infrastructure service 1
“with the model running on NVIDIA Dynamo, organizations can serve up to 8x more users with 85% lower latency.”
1 more mentioned or evaluated
kernel & quantization library “We use ModelOpt AutoQuantize (AutoQuant) to replace hand-authored layer rules with a sensitivity-constrained mixed-precision search.”
evaluation 30
filed at the root 11
“The benchmark side (question store, n-gram index, and embedding index) is built once and shared by all runs.”
“We evaluate MGSM (Shi et al., 2022) using 8-shot native chain-of-thought prompting in Spanish, German, French, Chinese, Japanese, and…”
“All models were evaluated under the same evaluation settings including agentic resources (CPU compute and timeouts, tools, search APIs)…”
“Judge gate semantic overlap≥0.75 or verbatim overlap≥0.5”
“It is important to stratify test cases along with capability partition to assess quantization damage properly.”
“Table 35 stratifies 1,470 MRCR-128k tasks at three repetitions to separate the retrieval from decoding noise.”
“As negatives we add unmodified questions from a different benchmark in the same domain, chosen to be hard to tell apart from the target”
“Chunking 512-byte windows with 256-byte stride (sample and question)”
“while tracking the failure of a model to terminate as it runs to the context wall”
“The LLM augmentations are generated with a model different from the judge, so that the judge cannot recognize its own rewrites.”
“transform each with 13 augmentations, at four strengths (0.25, 0.5, 0.75, 0.9)”
benchmark 4
“using EvalPlus-sanitized variants (Liu et al., 2023).”
“GPQA Diamond is evaluated using 5-shot chain-of-thought prompting.”
“MBPP-Sanitized uses 3-shot prompting”
“while HumanEval uses zero-shot prompting.”
evaluation harness 3
“All evaluation results reported for Nemotron 3.5 Super 120B-A12B Base and Lightning 3.5 30B-A3B Base were collected via NeMo Evaluator SDK”
“The published recipes specify the benchmark-specific containers, prompts, inference parameters, parser configurations, and scoring…”
“were collected via NeMo Evaluator SDK and NVIDIA’s open-source container of LM Evaluation Harness”
judge 8
“Judge model Claude Opus 4.6, temperature 0”
“Passing both texts as JSON-escaped strings keeps their boundaries unambiguous even when the sample itself contains code, markup, or…”
“The same checkpoint then serves as a verifier, selecting one candidate using LLM-as-a-Verifier (LAV).”
“Each shortlisted pair of a sample and a benchmark question is verified by an LLM judge.”
“For each compared pair and criterion, we evaluate both presentation orders”
“These pairwise evaluations are organized using LAV’s Probabilistic Pivot Tournament (PPT).”
“other free-form answers require same-teacher, text-only equivalence judging with reported confidence≥ 0.90.”
“evaluates them against three criteria: specification adherence, output match, and error signal detection.”
averaging - mean@k; avg@k; pass@k; pass^k 3
“We additionally evaluate MATH-500 using 4-shot sampled generation with 32 generations per prompt and report pass@1.”
“We also use the mean of per rubric scores to evaluate responses instead of an overall score.”
“We report sampled pass@1 estimated from 32 generations per prompt.”
1 more mentioned or evaluated
“To measure the effect of mid-training, we ran identical SFT on the base checkpoint and on the mid-trained checkpoint and compared the two.”
other 2
filed at the root 1
“We identify the runaway loop by reading reasoning passages in postmortem”
1 more mentioned or evaluated
“We divide and conquer the degradation by ablating non-attention components in pre-MOPD models”
unfiled 22
“It uses a hybrid Mamba-Transformer architecture”
“The model combines interleaved Mamba-2, Mixture-of-Experts (MoE), and attention layers”
“The model combines interleaved Mamba-2, Mixture-of-Experts (MoE), and attention layers”
“processes the resulting (B−1)×K candidate lattice jointly with a small Transformer”
“Following the V-model methodology, iterative testing and validation at both unit and system levels are essential”
“convert Manipuri translations from Bengali script to Meitei Mayek”
“The sample is compared with every benchmark question using two signals.”
“Semantic overlap maximum cosine similarity over chunk pairs”
“Semantic overlap measures the embedding similarity between parts of the two texts.”
“generate executable SpatialClaw (Cho et al., 2026) demonstrations from existing video question–answer pairs, teaching the student to…”
“the block is committed from left to right, each slot selecting the candidate most compatible with its committed predecessor”
“The combination of these two quantities is a useful early-warning indicator.”
“truncation, the fraction of rollouts containing at least one model call hitting an output limit”
“Users are responsible for ensuring safe integration of this model, including implementing guardrails as well as other safety mechanisms…”
“We optimize only the weights of language-model GEMMs with enabled quantizers. All other parameters remain frozen”
“We replay the MoE expert-routing decisions recorded during rollout generation when evaluating the policy, reducing train–generation…”
“The quantization configuration and calibrated quantizer statistics are restored from the PTQ checkpoint rather than recalibrated during QAD.”
“We optimize only the weights of language-model GEMMs with enabled quantizers.”
“Overall, our results favor shorter QAD training with 100% SFT data and show that optimal training duration depends on the selected data…”
“Speculative decoding, FP8 weights, and FlashAttention reduced median translation cost”
“The model ships with open weights, open datasets and training recipes.”
“Applications should validate generated answers, code, and tool arguments, and provide appropriate safeguards and human oversight”
Sources
The curated source list for this model. "not fetched" means the URL is recorded but its text was not read in the current run.
| Source | Kind | Publisher | Status |
|---|---|---|---|
| developer.nvidia.com/blog/nvidia-nemotron-3-5-super-vl-powers-specializ… | official blog | NVIDIA Technical Blog | read |
| blogs.nvidia.com/blog/nemotron-3-5-super-vl | official blog | NVIDIA | read |
| research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3.5-Super-and-L… | technical report | NVIDIA | read |
| research.nvidia.com/labs/nemotron/Nemotron-3.5-Super-and-Lightning | vendor docs | NVIDIA | read |
| huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16 | model card | NVIDIA | read |
| build.nvidia.com/nvidia/nemotron-3.5-super-vl-120b-a12b/modelcard | vendor docs | NVIDIA | read |
| openrouter.ai/nvidia/nemotron-3.5-super-vl | vendor docs | OpenRouter | read |