| DeepSeek-V4.1-Flash-1017 core | — | — | — | — | — | — | — | End-to-end project development coreReinforcement learning with self-feedback used— | — |
| DeepSeek-V4.1-Flash core | SFT followed by reinforcement learning and on-policy distillation used— | — | Checkpoint merging for successive RL runs usedConcatenated routing replay usedEffort-conditioned subgroup reward centering usedReinforcement learning mentioned— | Reasoning-length penalty used— | — | Dynamic Post-Training Recipe Reconfiguration usedFull-Vocabulary On-Policy Distillation with Heterogeneous Teachers usedSFT–RL–On-Policy Distillation Post-Training Recipe used— | Asynchronous reinforcement learning coreCompute-node sharding into scale units coreSample-level dispatch defaultAsynchronous reinforcement learning on synthesized tasks usedBounding off-policy sample ratio usedCo-located rollout and training usedDecoupled sandbox and worker-container architecture for RL rollouts usedPer-dataset rollout concurrency limits usedReuse of persisted rollout state without re-prefilling usedSample-grained garbage collection usedScaffold-agnostic rollout control and trajectory normalization usedState-preserving suspension and resumption of rollouts usedToken-granular persistence of rollout state usedToken-level interruption of generation usedDiscarding early-returned short samples optionalBatch-level rollout dispatch not usedPrompt-level rollout dispatch not used— | Durable peer-mailbox communication coreCollaborative multi-agent construction of coding environments usedCrash-as-failure trajectory handling usedFailure replay and targeted reinforcement learning usedGenerating real-workflow agent environments from feedback and failure cases usedIndependent multi-agent testing and inspection of coding environments usedIsolated-container environment setup and packaging usedIterative agent-led environment repair and reverification usedLifecycle monitoring and quality re-auditing of RL tasks usedMocked-tool construction for real-world interfaces usedMulti-objective reinforcement-learning reward for agent collaboration usedFresh or snapshot-based teammate initialization optional— | — |
| GLM-5.3 core | post-training coreExtended post-training used— | — | SAO with compaction used— | — | — | — | — | Environment scaling for realistic training tasks used— | — |
| DeepSeek-V4-Flash-0731 used | Post-training optimization used— | — | — | — | — | — | — | Re-post-training for agentic capabilities used— | — |
| Hy3 used | Joint supervised fine-tuning and reinforcement learning used— | Supervised fine-tuning usedFine-tuning optional— | Group Relative Policy Optimization usedReinforcement learning used— | — | — | — | — | — | — |
| MiMo-V2.6-Flash core | — | SFT checkpoint intended as an RL starting point coreSelf-correction cold start usedSupervised fine-tuning on MiMo-generated task data used— | Asynchronous Group Relative Policy Optimization coreGroupwise Advantage Redistribution corePenalty Module coreDropping zero-advantage rollout groups defaultAdvantage shaping usedDomain-specific GRPO training usedDynamic sampling of informative rollout groups usedFlagged-token advantage penalty and masking usedFreezing the MoE router during reinforcement learning usedGroup-relative length penalty usedHierarchical penalty escalation usedQuality-weighted advantage redistribution usedReplay top-k/top-p sampling candidate sets during training usedRollout Routing Replay usedSingle-harness reinforcement learning usedZero-reward correction for externally supported trajectories used— | Groupwise Agentic Grading coreGroupwise Reward Synthesis coreAdversarial-agent reward-hacking audit usedBehavior rubrics usedFive-dimension comparative grading of passing patches usedMonitoring-only penalty strategy usedMultiplicative reward synthesis from test and rubric scores usedNegative rubric checks for unintended side effects usedOffline trajectory auditing for reward hacking usedRule-based or model-judged trajectory error detection usedSegment-level behavioral penalties usedSolution-quality rubrics usedVerifier cross-checking used— | — | Autonomous Student Rollouts usedDistillation on MiMo-Generated Data usedModel Distillation usedMulti-Prefix Multi-Teacher On-Policy Distillation usedPrefix-Conditioned On-Policy Distillation usedStandard Multi-Teacher On-Policy Distillation used— | Agent Loop coreAgent-centric rollout execution coreAsynchronous reinforcement learning coreHierarchical trajectory data organization corePayload Porter corePersistent multi-tenant harness pool corePredictive Rollout Dispatch coreAdaptive Rollout Concurrency usedAdaptive Rollout Scheduling usedBackground-thread offloading of blocking rollout work usedComposite early-stop strategy that zeroes reward and masks affected contexts usedDecoupled control plane and data plane usedDialogue prefix matching usedDuration- and acceptance-adjusted per-source oversampling usedHeterogeneous Agent Harnesses usedIncremental multimodal-delta transfer usedIsolated resettable sandboxes for agent rollouts usedMixed-task rollout infrastructure usedMultimodal metadata/payload separation usedPartial rollout usedPer-source priors for rollout length estimation usedSample Mixer usedSample replay usedTraining–inference consistency mechanisms usedAsynchronous group-wise grading optionalDeficit-corrected scheduling evaluated— | Agentic reinforcement learning coreContainer-level network isolation usedEnvironment hardening during reinforcement learning usedEnvironment preparation to prevent solution leakage usedMulti-harness reinforcement learning usedUnified trajectory representation for agentic RL used— | Agent-centric mid-training usedMid-training alignment data from reward-hacking cases used— |
| NVIDIA-Nemotron-3-Ultra-550B-A55B core | — | Light supervised fine-tuning on teacher-distribution data coreRejection sampling usedSupervised fine-tuning usedToken-budget-based SFT data blending used— | Abstention training usedDynamic abstention-reward calibration usedGroup Relative Policy Optimization usedIcePop usedNegative-advantage penalties for malformed reasoning and tool calls usedPivotRL usedPPO-style policy-ratio clipping usedReinforcement learning usedReinforcement learning from verifiable rewards used— | Generative Reward Model usedLength penalty usedPrinciple-conditioned generative reward model used— | Reinforcement Learning from Human Feedback (RLHF) used— | Multi-Teacher On-Policy Distillation coreAsynchronous Multi-Teacher On-Policy Distillation usedBehavior–Proximal Policy Decoupling usedIcePop Token-Level Loss Masking usedOff-Policy Distillation not used— | One-step off-policy asynchronous reinforcement learning coreAsynchronous reinforcement learning usedOn-policy rollout generation used— | Concurrent multi-environment post-training usedMulti-environment reinforcement learning used— | Continued pretraining usedLong-context extension via continued pretraining used— |
| DeepSeek-V4-Flash core | — | — | Group Relative Policy Optimization used— | Direct RL optimization of a generative reward model usedGenerative Reward Model used— | — | Multi-Teacher On-Policy Distillation coreOn-Policy Distillation used— | — | — | — |
| Alpamayo 1.5 / 2 Super core | Multi-stage training strategy coreThree-stage VLA training: action-modality injection, CoC reasoning SFT, and RL post-training core— | Supervised fine-tuning on Chain of Causation data core— | RL-based post-training framework with complementary rewards coreGroup Relative Policy Optimization usedReinforcement learning used— | Reasoning reward from LRM feedback usedReasoning-action consistency reward usedSafety reward usedTrajectory-quality reward used— | — | — | — | — | Physical-AI-focused pretraining used— |
| DeepSeek-V3.2 core | Scalable reinforcement learning post-training core— | Prompt-based cold start for reasoning with tool use usedReasoning-data system prompt used— | Group Relative Policy Optimization coreKeep Routing coreMixed-domain reinforcement learning coreImportance-sampling-corrected unbiased KL estimate usedKeep Sampling Mask usedOff-Policy Sequence Masking usedReinforcement learning used— | Generative Reward Model usedLanguage-consistency reward usedLength penalty usedRule-based outcome reward used— | — | Specialist Distillation used— | — | Generate-verify-refine loop usedLarge-scale reinforcement learning on synthetic agentic data usedTraining agentic policies with real-world tools usedRL on synthetic agentic tasks in non-thinking mode evaluatedRL restricted to search and code environments evaluated— | Continued pretraining used— |
| DeepSeek-V4-Flash-Vision-Exp core | Post-training optimization used— | — | — | — | — | — | — | — | Continued training for visual understanding core— |
| DeepSeek-V4-Pro core | — | Specialist training usedSupervised fine-tuning used— | Group Relative Policy Optimization usedReinforcement learning used— | Direct RL optimization of a generative reward model usedGenerative Reward Model used— | — | Multi-Teacher On-Policy Distillation coreOn-Policy Distillation core— | Preemptible rollout service with write-ahead logging used— | — | Mid-training used— |
| DeepSeek-V4-Pro-0813 used | Post-training optimization used— | — | — | — | — | — | — | — | — |
| FLUX 3 Action core | — | Frozen Qwen3-VL-4B instruction encoder coreFrozen video encoder for camera frames coreAction-focused second-stage training usedFine-tuning for video-game and simulated-drone control usedRobot-specific fine-tuning with a small demonstration set usedFine-tuning action heads for a new embodiment optional— | — | — | — | — | — | — | — |
| GLM-5.2 used | — | Rejection sampling used— | Direct Double-sided Importance Sampling usedGroup Relative Policy Optimization usedGroup-wise policy optimization usedIcePop used— | Hybrid reward system usedMulti-level reward formulation used— | — | On-Policy Cross-Stage Distillation used— | Asynchronous Agent Reinforcement Learning algorithms usedAsynchronous reinforcement learning usedMulti-Task Rollout Orchestrator usedSLIME usedToken-in-Token-out (TITO) used— | — | — |
| GR00T N1.7 used | Post-training with real or synthetic data optional— | Fine-tuning video generation models on robot teleoperation data usedFreeze language and visual backbones during fine-tuning usedFreeze the language component of the vision-language backbone usedLow-Rank Adaptation usedPost-training on the Bridge Dataset in SimplerEnv usedPost-training on the DROID Dataset usedPost-training on the Fractal Dataset in SimplerEnv usedPost-training on the LIBERO Dataset usedSingle-embodiment post-training fine-tuning usedFine-tuning on custom embodiments with LeRobot-format datasets optional— | — | — | — | — | — | — | Human-video pretraining used— |
| Gemma 4 31B mentioned | — | Fine-tuning mentioned— | — | — | — | — | — | — | — |
| Inkling core | Post-training on diverse domains used— | Low-Rank Adaptation coreSafety training to an internal specification usedSFT bootstrapping with synthetic data used— | Reinforcement learning usedRL for calibration with proper scoring rules usedRL with rubric and claims graders used— | Abstention-aware factual QA reward used— | — | Distillation-Based Post-Training Data Generation used— | Asynchronous reinforcement learning core— | — | — |
| Kimi K3 core | Three-stage post-training with SFT, RL, and multi-teacher on-policy distillation core— | Supervised fine-tuning used— | Multi-domain, multi-effort reinforcement learning coreDomain-specialized RL experts used— | Agentic Generative Reward Model usedPublic and hidden verifier pairing usedWeb-development reward with deterministic checks and model judging used— | Budget-based verbosity control used— | Multi-Teacher On-Policy Distillation corePer-Token On-Policy Distillation Reward with Clipping used— | Co-located million-token agentic RL system with resumable sandboxes usedCo-located rollout and training usedPartial rollout used— | Training on verifiable problems in agentic environments coreAutonomous Execution Tasks usedMock applications for personal-assistant RL usedUnified white-box RL environment used— | — |
| Kolibri-1 core | Matched-quantity context masking coreDataset-specific reference groups for reasoning-effort labeling usedReasoning-effort labeling usedSeeded soft effort-label sampling around bucket boundaries used— | German-language reasoning post-training coreAssistant-only loss usedContext-grounded abstention training usedFine-tuning a specialist model for each dataset cluster usedInclude English and German safety datasets in SFT usedLong-context post-training with document question answering usedMask loss on incorrect tool calls and terminal actions usedMask loss on invalid or repeated tool calls usedSafety refusal training with appropriate partial answers usedSFT refusal training for PII extraction attacks usedSupervised fine-tuning usedLong-context SFT data ablation evaluatedLoss masking in supervised fine-tuning evaluatedReasoning-effort assignment in SFT evaluatedShortened supervised fine-tuning ablation recipe evaluated— | Difficulty-bucket and pass-rate curriculum coreGroup-relative advantage estimation coreMerlin and Morgana masking algorithms coreDynamic sampling of informative rollout groups usedLarge-scale reinforcement learning for long-horizon tasks usedMask re-rendered context during training usedOmitting advantage normalization used— | Environment-specific output-format reward signals coreMultiplicative aggregation of format signals corep-norm aggregation of format signals coreReinforcement Learning with Verifiable Rewards coreRole-centered reward baselines coreRole-conditioned Merlin–Arthur rewards coreFormat reward conditioned on positive environment reward usedHallucination mitigation through abstention and context-sensitive RL usedKillable, time-limited verifier process pool usedLanguage-consistency reward usedMerlin–Arthur correctness and abstention reward usedMerlin–Arthur hallucination and abstention example construction usedMultiplicative p-norm format reward usedReasoning-to-action verification usedRegression-adjusted test-pass reward usedTest-case pass-fraction reward used— | — | On-Policy Self-Distillation usedReasoning-Prefill Distillation used— | Asynchronous reinforcement learning coreOne rollout worker per non-controller node coreRouting sequences to the same inference replica coreAsynchronous batching of parallel retrieval queries usedCapacity allocation balancing trainer and inference throughput usedDistributed rollout workers usedIsolated resettable sandboxes for agent rollouts usedMaximum-staleness replay-buffer eviction usedNon-blocking weight synchronization to inference usedOverlapped RDMA weight-transfer writes usedPreserving the environment sampling mix usedRDMA weight distribution through one replica per node usedReplay-buffer shared-prefix deduplication usedRouted-Expert Replay used— | Environment sampling proportional to sampling and curriculum weights coreMerlin–Arthur protocol corePrompt sampling by difficulty bucket and evaluation recency coreRandomization of RL environments and prompts coreRetrieval-augmented question-answering training coreWeighted environment sampling coreAgentic retrieval environments for RAG training usedGerman-language RL training environments usedHarness randomization for agentic training usedLong-context question-answering RL environment usedMorgana adversarial context hiding usedPer-episode configuration randomization usedPer-group randomization of system prompt, coding harness, and user request usedPrompt typo injection usedRandomized preservation of reasoning traces across turns usedRandomized reasoning-effort level per rollout group usedRandomized retrieval setup and task language usedRandomized training environments for real-world robustness usedRL environments for political balance, copyright, and safety usedSelf-hosted user simulator usedSeparate abstention-only environment for unanswerable questions usedTool-calling and terminal-environment post-training usedTraining environments that simulate real-world use cases used— | Long-context extension via continued pretraining used— |
| LFM3-1.7B core | Three-phase audio model training coreThree-stage post-training pipeline coreFull post-training using TRL optional— | Supervised fine-tuning coreCross-category integration of multilingual samples in SFT data usedEnd-to-end fine-tuning usedFine-tuned skill monitor for robot recovery usedJoint multimodal supervised fine-tuning usedRank-32 Low-Rank Adaptation usedTask-specific fine-tuning for wind-turbine blade defect detection usedFrozen-backbone projection-layer adaptation optionalLocal fine-tuning for specialization optionalSingle-GPU fine-tuning optional— | Mixed-domain reinforcement learning usedSFT and GRPO for domain specialization used— | Length penalty used— | Length-normalized direct preference alignment coreOffline direct preference alignment coreContrastive Learning from AI Revisions (CLAIR) used— | Multi-Teacher On-Policy Distillation used— | — | Agentic post-training usedAgentic reinforcement learning used— | Joint text-and-audio mid-training usedLong-context extension via continued pretraining usedMid-training usedMid-training with lower-quality SFT datasets usedMultimodal mid-training usedContinued pretraining optional— |
| Laguna-S-2.1 core | Two-stage post-training pipeline (SFT then RL) core— | Evaluation-based early stopping during SFT usedFrozen-network warmup for new special tokens usedInstruction-following fine-tuning usedSFT bootstrapping with synthetic data used— | CISPO with length-weighted leave-one-out group-relative advantages coreAgentic RL task mix with a shared tool-use API usedRLCEF usedTool-error step penalty usedMoonlight scaling optional— | Binary task verifier coreDeterministic reward-checking chain coreSparse terminal-verifier reward corePer-token tool-error reward shaping used— | — | — | Blocking in-flight rollout steps during weight updates usedCapping trajectory staleness usedClosed-loop multi-turn synthetic-data rollout usedExact rollout-to-production chat-template alignment assertion usedLong-horizon rollout budgets usedMulti-harness rollouts usedRL sandboxing with selective network blocking and artifact caching usedToken-in-Token-out (TITO) used— | Agentic reinforcement learning coreProduction-harness-matched reinforcement learning coreMulti-harness trajectory training with native behavior preservation usedSeed-based terminal-task generation used— | Long-context extension via continued pretraining used— |
| LingBot-VLA 2.0 core | — | — | — | — | — | Dual-Query Distillation core— | — | — | — |
| MiMo-V2.5 core | — | Cold-started supervised fine-tuning model usedSupervised fine-tuning used— | Group Relative Policy Optimization usedReinforcement learning usedRollout Routing Replay used— | Outcome Reward Model usedRule-based verifiers usedTest-difficulty-driven code reward used— | — | Multi-Teacher On-Policy Distillation coreOn-Policy Distillation used— | Continuous rollout with asynchronous rewards and early termination usedData re-sampling strategy usedData Scheduler usedPartial rollout usedSeamless Rollout Engine usedStaleness-aware truncated importance sampling usedTool Manager usedToolbox used— | Agentic post-training usedAgentic reinforcement learning used— | — |
| MiMo-V2.5-Pro core | — | Cold-started supervised fine-tuning model usedSupervised fine-tuning used— | Reinforcement learning used— | Rule-based verifiers usedTest-difficulty-driven code reward used— | — | Multi-Teacher On-Policy Distillation core— | Continuous rollout with asynchronous rewards and early termination usedData re-sampling strategy usedSeamless Rollout Engine used— | Agentic post-training usedAgentic reinforcement learning used— | — |
| MiMo-V2.6-Pro core | — | SFT checkpoint intended as an RL starting point coreSelf-correction cold start usedSupervised fine-tuning on MiMo-generated task data used— | Penalty Module coreDropping zero-advantage rollout groups defaultAdvantage shaping usedAsynchronous Group Relative Policy Optimization usedDomain-specific GRPO training usedDynamic sampling of informative rollout groups usedFlagged-token advantage penalty and masking usedFreezing the MoE router during reinforcement learning usedGroup-relative length penalty usedGroupwise Advantage Redistribution usedHierarchical penalty escalation usedQuality-weighted advantage redistribution usedReplay top-k/top-p sampling candidate sets during training usedRollout Routing Replay usedSingle mixed-domain RL run usedSingle-harness reinforcement learning usedZero-reward correction for externally supported trajectories used— | Groupwise Agentic Grading coreAdversarial-agent reward-hacking audit usedBehavior rubrics usedFive-dimension comparative grading of passing patches usedGroupwise Reward Synthesis usedMonitoring-only penalty strategy usedMultiplicative reward synthesis from test and rubric scores usedNegative rubric checks for unintended side effects usedOffline trajectory auditing for reward hacking usedRule-based or model-judged trajectory error detection usedSegment-level behavioral penalties usedSolution-quality rubrics usedVerifier cross-checking used— | — | Autonomous Student Rollouts usedDistillation on MiMo-Generated Data usedModel Distillation usedMulti-Prefix Multi-Teacher On-Policy Distillation usedPrefix-Conditioned On-Policy Distillation usedStandard Multi-Teacher On-Policy Distillation used— | Agent Loop coreAgent-centric rollout execution coreAsynchronous reinforcement learning coreHierarchical trajectory data organization corePayload Porter corePersistent multi-tenant harness pool corePredictive Rollout Dispatch coreAdaptive Rollout Concurrency usedAdaptive Rollout Scheduling usedBackground-thread offloading of blocking rollout work usedComposite early-stop strategy that zeroes reward and masks affected contexts usedDecoupled control plane and data plane usedDialogue prefix matching usedDuration- and acceptance-adjusted per-source oversampling usedHeterogeneous Agent Harnesses usedIncremental multimodal-delta transfer usedIsolated resettable sandboxes for agent rollouts usedMixed-task rollout infrastructure usedMixing tasks and harnesses in one RL batch usedMultimodal metadata/payload separation usedPartial rollout usedPer-source priors for rollout length estimation usedSample Mixer usedSample replay usedTraining–inference consistency mechanisms usedAsynchronous group-wise grading optionalDeficit-corrected scheduling evaluated— | Agentic reinforcement learning coreContainer-level network isolation usedEnvironment hardening during reinforcement learning usedEnvironment preparation to prevent solution leakage usedMulti-agent task decomposition and coordination usedMulti-harness reinforcement learning usedUnified trajectory representation for agentic RL used— | Agent-centric mid-training usedMid-training alignment data from reward-hacking cases used— |
| MiniMax-M3 used | — | — | — | — | — | — | — | Multi-turn collaboration simulator used— | — |
| Mistral-Large-4 core | — | Assistant-only loss usedFreeze the vision encoder during pretraining and unfreeze for SFT usedLow-Rank Adaptation usedSupervised fine-tuning usedFine-tuning optionalFine-tuning on traces from Mistral Large 4 optional— | CISPO-style REINFORCE objective coreImportance weighting for asynchronous policy lag coreKeep importance-weighted tokens in the objective corePer-token weight normalization by rollout group coreStop-gradient upper-clipped importance weighting coreDropping zero-advantage rollout groups usedGroup-based rollout sampling usedGroup-centered batch-wide advantage normalization usedGroup-relative advantage estimation usedMiMo-style segment-level advantage correction usedOff-policy drift reduction usedReinforcement learning usedReinforcement-learning pipeline usedSFT checkpoint initialization for RL used— | Per-source composite reward functions coreConservative failure attribution defaultAdversarial-agent reward-hacking audit usedComposable multi-method reward verification usedExploit-neutralization and regression-free patch reward usedFail-to-pass test verifier construction usedGroup-relative length bonus usedPairwise rewards with groupwise penalties usedQuantized group-relative rollout-length bonus usedReward-scale calibration usedScientific-tool answer cross-checking usedReinforcement Learning with Verifiable Rewards optional— | — | Multi-Teacher On-Policy Distillation usedMultiple-Teacher Distillation usedRemote Teacher Infrastructure usedThree-Teacher Distillation for Reasoning-Effort Modes used— | Asynchronous reinforcement learning coreShared environment implementation across RL, evaluation, and synthetic data generation coreActor-pool orchestration for asynchronous RL usedAdaptive Rollout Concurrency usedAutoscaling rollout actors usedAutoscaling supporting services usedBackpressure- and concurrency-controlled rollout admission usedCentralized trajectory bin-packing usedDiscarding infrastructure-failure rollouts in RL usedExcluding auxiliary environment-model calls from training usedFixed-usable-trajectory-count batching usedLow-staleness optimization for long-trajectory RL usedPause and resume in-flight rollouts across policy updates usedPer-step NCCL weight updates usedPrioritizing sibling rollouts from started groups usedProvider failover routing usedRemote teacher hidden-state transfer with trainer-side logit reconstruction usedShared composable interface for RL tasks usedSubmission-only termination for environments with a submit tool optional— | Common agent-environment interaction loop coreTerminal-task difficulty calibration using solver pass rates used— | — |
| MolmoAct 2 core | post-training coreThree-stage VLA training pipeline corePost-training for multi-embodiment robot control usedSpecialize-then-rehearse training used— | Depth-token noise during fine-tuning coreEmbodiment-specific fine-tuning coreFull-model fine-tuning coreEmbodied specialization stage usedFine-tuning on target-embodiment demonstrations usedFine-tuning on the combined LIBERO training suites usedLow-Rank Adaptation usedRobot-only fine-tuning usedSeparate single-task policies for bimanual YAM tasks usedFine-tuning for a target robot embodiment or benchmark optionalAction-expert-only fine-tuning evaluatedFlow-loss updates through action-expert conditioning not usedFreeze added-token input embeddings during embodiment fine-tuning not used— | — | — | — | — | — | — | Mid-training coreMid-training on the MolmoAct Dataset used— |
| NVIDIA-Nemotron-3.5-Lightning-30B-A3B core | Specialized teacher training used— | Capability-targeted second-stage supervised fine-tuning usedLow-Rank Adaptation usedMask tool responses from the SFT loss usedRandom-budget truncation of reasoning traces usedSingle-stage supervised fine-tuning usedSupervised fine-tuning usedSupervised fine-tuning at 256k sequence length usedTwo-stage supervised fine-tuning usedUse 100% SFT data for final QAD usedExperimental agentic SFT blend evaluatedFine-tuning with demographically balanced datasets mentioned— | Abstention training usedAsynchronous Group Relative Policy Optimization usedClipped-range importance weighting with stale-token masking usedEnd-to-end GRPO on agentic tasks usedGroup Relative Policy Optimization usedGRPO with masked importance sampling usedLength-based RL reward adjustment usedReinforcement learning from verifiable rewards used— | Auxiliary tool-use reward usedBinary answer-correctness reward usedContinuous tracking-quality reward usedCorrect-response-only soft length preference usedDeterministic task-verifier rewards usedMatched think-block format validation usedReasoning-length penalty usedReinforcement Learning with Verifiable Rewards used— | Reinforcement Learning from Human Feedback (RLHF) coreFixed-reference response comparison used— | Token-Local Distillation Update coreDistillation from a Larger Nemotron 3 Model usedFrozen Student Self-Anchor usedMulti-Teacher On-Policy Distillation usedProximal-Teacher Trust-Region Policy Distillation usedTwo-Stage Multi-Teacher On-Policy Distillation used— | Asynchronous reinforcement learning usedIndependent parallel agent rollouts usedOn-policy rollout generation usedOne-step-stale asynchronous response generation usedTeacher-generated rollout trajectories used— | Multi-environment reinforcement learning coreFinance-agent environment with consistent tools across training stages usedHarness-optimized training usedTraining agentic teachers on sandboxed executable tasks used— | Long-context extension via continued pretraining coreMid-training coreIndic continued pretraining usedLong-context recovery training usedSoftware-engineering mid-training used— |
| NVIDIA-Nemotron-3.5-Super-VL-120B-A12B core | Specialized teacher training used— | Two-stage supervised fine-tuning coreCapability-targeted second-stage supervised fine-tuning usedMask tool responses from the SFT loss usedPost-training on organization-specific data usedRandom-budget truncation of reasoning traces usedSingle-stage supervised fine-tuning usedSupervised fine-tuning usedUse 100% SFT data for final QAD usedDomain-specific post-training optionalLow-Rank Adaptation optionalExperimental agentic SFT blend evaluated— | Abstention training usedAsynchronous Group Relative Policy Optimization usedClipped-range importance weighting with stale-token masking usedEnd-to-end GRPO on agentic tasks usedLength-based RL reward adjustment usedReinforcement learning usedReinforcement learning from verifiable rewards used— | Auxiliary tool-use reward usedBinary answer-correctness reward usedContinuous tracking-quality reward usedCorrect-response-only soft length preference usedDeterministic task-verifier rewards usedMatched think-block format validation usedReasoning-length penalty usedReinforcement Learning with Verifiable Rewards used— | Reinforcement Learning from Human Feedback (RLHF) coreFixed-reference response comparison used— | Token-Local Distillation Update coreDistillation from a Larger Nemotron 3 Model usedDistillation from Nemotron 3 Ultra Base usedFrozen Student Self-Anchor usedMulti-Teacher Capability Consolidation usedMulti-Teacher On-Policy Distillation usedProximal-Teacher Trust-Region Policy Distillation usedTwo-Stage Multi-Teacher On-Policy Distillation used— | Independent parallel agent rollouts usedOn-policy rollout generation usedOne-step-stale asynchronous response generation usedTeacher-generated rollout trajectories used— | Finance-agent environment with consistent tools across training stages usedTraining agentic teachers on sandboxed executable tasks used— | Long-context extension via continued pretraining coreMid-training coreContinued pretraining with natural and synthetic data usedIndic continued pretraining usedLong-context recovery training usedSoftware-engineering mid-training used— |
| OpenVLA / OpenVLA-OFT core | — | Fine-tuning a vision-language model to predict robot actions coreFine-tuning all model parameters including the vision backbone coreOptimized Fine-Tuning coreVision-encoder fine-tuning during VLA training coreEnd-to-end fine-tuning usedFine-tuning with setup-specific demonstrations usedFull-model fine-tuning usedImitation learning usedLow-Rank Adaptation usedParameter-efficient fine-tuning usedSupervised fine-tuning on target-task demonstrations usedFine-tuning with demonstration data for new tasks and robot setups optionalFine-tuning on BridgeData V2 only evaluatedFreeze the vision encoder during fine-tuning evaluatedLast-layer and token-embedding fine-tuning evaluatedSandwich fine-tuning evaluated— | — | — | — | — | — | — | — |
| Qwen3.5-397B-A17B core | Scalable RL at Agent Scale coreThree-stage Thinker post-training strategy used— | Lightweight speaker fine-tuning used— | GSPO usedInteraction-Aligned RL usedProgressively complex task distributions used— | — | Direct Preference Optimization (DPO) used— | On-Policy Distillation usedSpecialist Distillation used— | Asynchronous RL framework for large-scale agent scaffolds and environment orchestration used— | — | Continued pretraining used— |
| Qwen3.6-35B-A3B core | Scalable RL at Agent Scale core— | — | — | — | — | — | — | — | — |
| SmolVLA core | Pretraining on general manipulation data coreTask-specific post-training core— | Fine-tuning usedFrozen-backbone action-expert fine-tuning usedImitation learning usedMultitask fine-tuning used— | — | — | — | — | Asynchronous reinforcement learning evaluated— | — | — |
| VLA Foundry (Foundry-VLA-1.7B) used | — | Task-specific fine-tuning of a multitask policy evaluated— | — | — | — | — | — | — | Sequential LLM-to-VLM-to-VLA training pipeline used— |
| X-VLA used | — | Embodiment-specific fine-tuning usedJoint policy adaptation usedPeriodic policy training with failure-targeted demonstration collection usedPrompt warm-up with the pretrained backbone frozen usedJoint fine-tuning across multiple embodiments evaluatedLow-Rank Adaptation evaluatedHeterogeneous Low-Rank Adaptation not used— | — | — | — | — | — | — | — |
| Xiaomi-Robotics-1 core | Pre-training followed by post-training for robot policies core— | — | — | — | — | — | — | — | — |
| gpt-oss-120b core | — | Adversarial fine-tuning usedInstruction hierarchy training usedFine-tuning optional— | Chain-of-thought reinforcement learning coreHelpful-only adversarial reinforcement learning usedIncremental reinforcement learning used— | — | Deliberative alignment used— | — | — | Agentic tool-use post-training usedPython-tool use during chain-of-thought used— | — |
| π0.5 core | Pre-training followed by post-training for robot policies corepost-training usedTask-specific post-training used— | Adaptation of vision-language models using robot data usedFine-tuning on downstream robot tasks usedFine-tuning with high-quality data usedFull-model fine-tuning usedFine-tuning optionalLow-Rank Adaptation optionalTraining a downstream policy only on high-quality task data evaluated— | — | — | — | — | — | — | Broad robot-data pretraining used— |