Model techniques map
Taxonomypost-training

taxonomy area · pipeline stage 05

post-training

267 methods filed at this node or below it, from the sources of 24 models.

post-training

Matching aids for the classifier: what happens to a pretrained checkpoint.

In this branch 267

Everything filed at this node or below it, with one collapsible heading per child node.

filed here 10

Scalable RL at Agent Scale core · 3 sources · 3 quotes
Scalable reinforcement learning post-training core · 2 sources · 2 quotes
SFT followed by RL and on-policy distillation default · 1 source · 2 quotes
Extended post-training used · 1 source · 1 quote
Joint SFT and RL used · 1 source · 1 quote
Post-training on diverse domains used · 1 source · 1 quote
Post-training optimization used · 1 source · 1 quote
Three-stage Thinker post-training strategy used · 1 source · 1 quote

supervised fine-tuning 24

LoRA fine-tuning core · 2 sources · 2 quotes
Light SFT on teacher-distribution data core · 1 source · 2 quotes
SFT checkpoint for RL research core · 1 source · 1 quote
Supervised fine-tuning used · 14 sources · 17 quotes
Rejection sampling used · 2 sources · 2 quotes
Self-correction cold start used · 2 sources · 2 quotes
SFT bootstrapping with synthetic data used · 2 sources · 2 quotes
Adversarial fine-tuning used · 1 source · 1 quote
Autoregressive fine-tuning used · 1 source · 1 quote
Cold-started SFT model used · 1 source · 1 quote
Evaluation-based early stopping used · 1 source · 1 quote
Instruction hierarchy training used · 1 source · 1 quote
Instruction-following fine-tuning used · 1 source · 1 quote
Lightweight speaker fine-tuning used · 1 source · 1 quote
Prompt-based cold start for tool use used · 1 source · 1 quote
Reasoning-data system prompt used · 1 source · 1 quote
SFT on MiMo-generated task data used · 1 source · 1 quote
Specialist-model training used · 1 source · 1 quote
Token-budget-based SFT data blending used · 1 source · 1 quote
Fine-tuning optional · 3 sources · 3 quotes

reinforcement learning algorithm 62

Group Relative Policy Optimization core · 15 sources · 17 quotes
Groupwise Advantage Redistribution core · 3 sources · 4 quotes
Chain-of-Thought Reinforcement Learning core · 2 sources · 2 quotes
Mixed Reinforcement Learning core · 2 sources · 2 quotes
Effort-Conditioned Reinforcement Learning core · 1 source · 1 quote
Keep Routing core · 1 source · 1 quote
Penalty Module core · 1 source · 1 quote
Dropping All-Zero-Advantage Groups default · 1 source · 1 quote
Reinforcement Learning used · 11 sources · 12 quotes
IcePop used · 2 sources · 3 quotes
Keep Sampling Mask used · 2 sources · 2 quotes
Off-Policy Sequence Masking used · 2 sources · 3 quotes
Reinforcement Learning Post-Training used · 2 sources · 2 quotes
Rollout Routing Replay used · 2 sources · 3 quotes
Unbiased KL Estimate used · 2 sources · 3 quotes
Abstention Training used · 1 source · 1 quote
Advantage Shaping used · 1 source · 1 quote
Agentic RL Task Mix used · 1 source · 1 quote
Concatenated Routing Replay used · 1 source · 1 quote
Derived-Latency Penalty used · 1 source · 1 quote
Direct Double-Sided Importance Sampling used · 1 source · 2 quotes
Domain-Specialized RL Experts used · 1 source · 1 quote
Domain-Specific GRPO Training used · 1 source · 1 quote
Dynamic Abstention-Reward Calibration used · 1 source · 1 quote
Dynamic Sampling used · 1 source · 1 quote
Effort-Dependent Length Penalty used · 1 source · 1 quote
Flagged-Token Masking and Penalty used · 1 source · 1 quote
Group-Relative Length Penalty used · 1 source · 1 quote
Group-Wise Policy Optimization used · 1 source · 1 quote
GRPO with Masked Importance Sampling used · 1 source · 1 quote
GSPO used · 1 source · 1 quote
Hierarchical Penalty Escalation used · 1 source · 1 quote
Incremental Reinforcement Learning used · 1 source · 1 quote
Interaction-Aligned Reinforcement Learning used · 1 source · 1 quote
Keep Candidate-Set Replay used · 1 source · 1 quote
Large-Scale Reinforcement Learning used · 1 source · 1 quote
Per-Token Tool-Error Step Penalty used · 1 source · 1 quote
Pivot Reinforcement Learning used · 1 source · 2 quotes
PPO-Style Policy-Ratio Clipping used · 1 source · 1 quote
Progressively Complex Task Distributions used · 1 source · 1 quote
Quality-Weighted Advantage Redistribution used · 1 source · 1 quote
Reinforcement Learning Training used · 1 source · 1 quote
RL with Rubric and Claims Graders used · 1 source · 1 quote
SAO with Compaction used · 1 source · 1 quote
Single Mixed Reinforcement-Learning Run used · 1 source · 1 quote
Single-Harness Reinforcement Learning used · 1 source · 1 quote
Moonlight Scaling optional · 1 source · 1 quote

reward modelling 35

Groupwise Agentic Grading core · 3 sources · 4 quotes
Groupwise Reward Synthesis core · 3 sources · 4 quotes
Binary task verifier core · 1 source · 1 quote
Binary terminal-verifier reward core · 1 source · 1 quote
Deterministic chain of checkers core · 1 source · 1 quote
Generative Reward Model used · 6 sources · 7 quotes
Adversarial screening used · 2 sources · 2 quotes
Length-adjusted RL reward used · 2 sources · 2 quotes
Verifier cross-checking used · 2 sources · 2 quotes
Abstention-aware reward for factual QA used · 1 source · 1 quote
Agentic Generative Reward Model used · 1 source · 1 quote
Behavior rubrics used · 1 source · 1 quote
Collaboration bonus used · 1 source · 1 quote
Hack-agent screening used · 1 source · 1 quote
Hybrid reward system used · 1 source · 1 quote
Language consistency reward used · 1 source · 1 quote
Monitoring-only penalty strategy used · 1 source · 1 quote
Multi-level reward formulation used · 1 source · 1 quote
Multiplicative reward synthesis used · 1 source · 1 quote
Outcome Reward Model used · 1 source · 1 quote
Per-token tool-error reward shaping used · 1 source · 1 quote
Public and hidden verifier pairing used · 1 source · 1 quote
Rule-based outcome reward used · 1 source · 1 quote
Rule-based verifier used · 1 source · 1 quote
Segment-level behavioral penalties used · 1 source · 1 quote
Solution rubrics used · 1 source · 1 quote
Test-difficulty-driven code reward used · 1 source · 1 quote
Training-time trajectory auditing used · 1 source · 1 quote

preference optimization 4

Reinforcement Learning from Human Feedback used · 3 sources · 3 quotes
Budget-based verbosity control used · 1 source · 1 quote
Deliberative alignment used · 1 source · 1 quote
Direct Preference Optimization used · 1 source · 1 quote

policy distillation 19

Multi-Teacher On-Policy Distillation core · 12 sources · 17 quotes
On-Policy Distillation core · 7 sources · 9 quotes
Specialist Distillation used · 3 sources · 3 quotes
Prefix-Conditioned On-Policy Distillation used · 2 sources · 5 quotes
Autonomous Student Rollouts used · 1 source · 2 quotes
Behavior–Proximal Policy Decoupling used · 1 source · 1 quote
IcePop Token-Level Loss Masking used · 1 source · 1 quote
Large-Scale On-Policy Distillation used · 1 source · 2 quotes
Model Distillation used · 1 source · 1 quote
Multi-Objective Policy Distillation used · 1 source · 1 quote
On-Policy Cross-Stage Distillation used · 1 source · 2 quotes
Per-Token On-Policy Distillation Reward used · 1 source · 1 quote
SFT–RL–On-Policy Distillation Pipeline used · 1 source · 1 quote
Teacher-Trajectory and SFT-History Reuse used · 1 source · 1 quote
Off-Policy Distillation not used · 1 source · 1 quote

rollout & RL infrastructure 76

Asynchronous reinforcement learning core · 6 sources · 7 quotes
Agent Loop core · 1 source · 1 quote
Agent-centric rollout execution core · 1 source · 1 quote
Harness Pool core · 1 source · 1 quote
Hierarchical trajectory data organization core · 1 source · 1 quote
Large-scale asynchronous RL core · 1 source · 1 quote
Payload Porter core · 1 source · 1 quote
Predictive Rollout Dispatch core · 1 source · 2 quotes
Sample-level dispatch default · 1 source · 1 quote
Partial rollout used · 3 sources · 5 quotes
Co-located RL training used · 2 sources · 2 quotes
Seamless Rollout Engine used · 2 sources · 2 quotes
SLIME used · 2 sources · 2 quotes
Token-in-token-out (TITO) used · 2 sources · 3 quotes
Adaptive Rollout Concurrency used · 1 source · 1 quote
Adaptive Rollout Scheduling used · 1 source · 1 quote
Asynchronous Agent RL algorithms used · 1 source · 1 quote
Asynchronous sample generation used · 1 source · 1 quote
Bounding the maximum off-policy ratio used · 1 source · 1 quote
Capping trajectory staleness used · 1 source · 1 quote
Closed-loop multi-turn rollout used · 1 source · 1 quote
Co-located long-context agentic RL system used · 1 source · 1 quote
Composite early-stop strategy used · 1 source · 1 quote
Compute-node sharding into scale units used · 1 source · 1 quote
Consistent configuration transitions used · 1 source · 1 quote
Data re-sampling strategy used · 1 source · 1 quote
Data Scheduler used · 1 source · 2 quotes
Decoupled control plane and data plane used · 1 source · 2 quotes
Dialogue prefix matching used · 1 source · 1 quote
Discarding early-returned short samples used · 1 source · 1 quote
Dynamic training recipe reconfiguration used · 1 source · 1 quote
Full-lifecycle monitoring of RL tasks used · 1 source · 1 quote
Heterogeneous Agent Harnesses used · 1 source · 1 quote
Incremental image transfer used · 1 source · 1 quote
Incremental multimodal-delta transfer used · 1 source · 1 quote
Isolated resettable rollout sandboxes used · 1 source · 1 quote
Long-horizon rollout budgets used · 1 source · 1 quote
Mixed-task rollout infrastructure used · 1 source · 1 quote
Multi-harness rollouts used · 1 source · 1 quote
Multi-Task Rollout Orchestrator used · 1 source · 1 quote
Multimodal rollout data handling used · 1 source · 1 quote
On-policy rollouts used · 1 source · 1 quote
Per-dataset rollout concurrency limiting used · 1 source · 1 quote
Per-source oversampling allocation used · 1 source · 1 quote
Persistent host-actor pools used · 1 source · 1 quote
Preemptible rollout service used · 1 source · 1 quote
Sample Mixer used · 1 source · 1 quote
Sample replay used · 1 source · 1 quote
Sample-grained garbage collection used · 1 source · 2 quotes
Scaffold-agnostic rollout control layer used · 1 source · 1 quote
Token-level interruption used · 1 source · 2 quotes
Tool Manager used · 1 source · 2 quotes
Toolbox used · 1 source · 2 quotes
Asynchronous group-wise grading optional · 1 source · 1 quote
Deficit-corrected scheduling evaluated · 1 source · 2 quotes
Batch-level rollout dispatch not used · 1 source · 1 quote
Prompt-level dispatch not used · 1 source · 1 quote

agentic post-training 31

Agentic reinforcement learning core · 4 sources · 4 quotes
Multi-environment reinforcement learning core · 1 source · 1 quote
Reinforcement learning in an agent harness core · 1 source · 1 quote
Agentic tool-use training used · 2 sources · 2 quotes
Environment hardening used · 2 sources · 2 quotes
Agentic post-training used · 1 source · 1 quote
Autonomous Execution Tasks (AET) used · 1 source · 1 quote
Concurrent multi-environment post-training used · 1 source · 1 quote
Container-level network isolation used · 1 source · 1 quote
Generate-verify-refine loop used · 1 source · 1 quote
Harness-optimized training used · 1 source · 1 quote
Multi-harness training used · 1 source · 3 quotes
Multi-turn collaboration simulator used · 1 source · 1 quote
Python tool use in chain-of-thought used · 1 source · 1 quote
Re-post-training for agentic capabilities used · 1 source · 1 quote
Repair-agent environment correction loop used · 1 source · 2 quotes
Seed-based terminal task generation used · 1 source · 1 quote

mid-training & continual pretraining 6

Continual pretraining used · 3 sources · 3 quotes
Agent-centric mid-training used · 1 source · 1 quote
Mid-training alignment data used · 1 source · 1 quote
Mid-training phase used · 1 source · 1 quote

By model

Which of this branch's techniques each model's own documents describe, and how strongly. Under each model: its strongest treatment anywhere in the branch.

Modelfiled heresupervised fine-tuningreinforcement learning algorithmreward modellingpreference optimizationpolicy distillationrollout & RL infrastructureagentic post-trainingmid-training & continual pretraining
DeepSeek-V4.1-Flash coreSFT followed by RL and on-policy distillation default—Autoregressive fine-tuning used—Effort-Conditioned Reinforcement Learning coreConcatenated Routing Replay usedDerived-Latency Penalty usedEffort-Dependent Length Penalty usedExponentially Decaying Token-Penalty Coefficient usedFailure Replay and Targeted Reinforcement Learning usedGroup-Relative Advantage Normalization Within Effort-Level Subgroups usedProgressive Scaling of RL Data, Tasks, and Rollouts usedRL Reward with Collaboration Bonus and Derived-Latency Penalty usedReinforcement Learning Post-Training mentioned—Collaboration bonus used——Large-Scale On-Policy Distillation usedOn-Policy Distillation usedSFT–RL–On-Policy Distillation Pipeline used—Asynchronous reinforcement learning framework coreSample-level dispatch defaultAsynchronous sample generation usedBounding the maximum off-policy ratio usedCo-located RL training usedCompute-node sharding into scale units usedConsistent configuration transitions usedDecoupling agent rollout into sandbox and worker container usedDiscarding early-returned short samples usedDynamic training recipe reconfiguration usedFull-lifecycle monitoring of RL tasks usedIncremental image transfer usedLarge-scale asynchronous RL in synthesized tasks usedPer-dataset rollout concurrency limiting usedPer-task upper bound on in-flight rollout samples usedPreemption-safe suspension and offloading of rollout execution usedSample-grained garbage collection usedScaffold-agnostic rollout control layer usedScaling RL along training compute and number of scaffolds usedToken-granularity persistence of rollout states usedToken-level interruption usedBatch-level rollout dispatch not usedPrompt-level dispatch not used—Repair-agent environment correction loop usedRepercussion signal for agent-crashed environments used——
DeepSeek-V4-Flash-0731 usedPost-training optimization used———————Re-post-training for agentic capabilities used——
NVIDIA-Nemotron-3-Ultra-550B-A55B core—Light SFT on teacher-distribution data coreRejection sampling usedSupervised fine-tuning usedToken-budget-based SFT data blending used—Abstention Training usedDynamic Abstention-Reward Calibration usedGroup Relative Policy Optimization usedIcePop usedNegative-Advantage Penalties for Malformed Reasoning and Tool Calls usedPivot Reinforcement Learning usedPPO-Style Policy-Ratio Clipping usedReinforcement Learning usedReinforcement Learning from Verifiable Rewards used—Generative Reward Model usedLength-adjusted RL reward usedPrinciple-conditioned Generative Reward Model used—Reinforcement Learning from Human Feedback used—Multi-Teacher On-Policy Distillation coreAsynchronous Multi-Teacher On-Policy Distillation usedBehavior–Proximal Policy Decoupling usedIcePop Token-Level Loss Masking usedMulti-Objective Policy Distillation usedOff-Policy Distillation not used—One-step off-policy asynchronous reinforcement learning coreAsynchronous reinforcement learning usedOn-policy rollouts used—Concurrent multi-environment post-training usedMulti-environment reinforcement learning from verifiable rewards used—Continual pretraining usedContinual pretraining for long-context extension used—
MiMo-V2.6-Flash core—SFT checkpoint for RL research coreSelf-correction cold start usedSFT on MiMo-generated task data used—Asynchronous Group Relative Policy Optimization coreGroupwise Advantage Redistribution corePenalty Module coreDropping All-Zero-Advantage Groups defaultAdvantage Shaping usedDomain-Specific GRPO Training usedDynamic Sampling usedFlagged-Token Masking and Penalty usedFreezing the MoE Router During Reinforcement Learning usedGroup-Relative Length Penalty usedHierarchical Penalty Escalation usedKeep Candidate-Set Replay usedQuality-Weighted Advantage Redistribution usedRollout Routing Replay usedSingle-Harness Reinforcement Learning usedZero-Reward Correction for Leaked or External Answers used—Groupwise Agentic Grading coreGroupwise Reward Synthesis coreAdversarial screening usedBehavior rubrics usedFive-dimension comparative grading of passing patches usedHack-agent screening usedMonitoring-only penalty strategy usedMultiplicative reward synthesis usedNegative checks for unintended side effects usedRule-based or model-judged trajectory error detection usedSegment-level behavioral penalties usedSolution rubrics usedTraining-time trajectory auditing usedVerifier cross-checking used——Autonomous Student Rollouts usedDistillation Fine-Tuning on MiMo-Generated Data usedModel Distillation usedMulti-Prefix Multi-Teacher On-Policy Distillation usedPrefix-Conditioned On-Policy Distillation used—Agent Loop coreAgent-centric rollout execution coreAsynchronous reinforcement learning coreHarness Pool coreHierarchical trajectory data organization corePayload Porter corePredictive Rollout Dispatch coreAdaptive Rollout Concurrency usedAdaptive Rollout Scheduling usedComposite early-stop strategy usedDecoupled control plane and data plane usedDialogue prefix matching usedHeterogeneous Agent Harnesses usedIncremental multimodal-delta transfer usedIsolated resettable rollout sandboxes usedMixed-task rollout infrastructure usedMultimodal rollout data handling usedOffloading blocking environment and tokenization work to background threads usedPartial rollout usedPer-source oversampling allocation usedPer-source priors for rollout sequence-length estimation usedPersistent host-actor pools usedSample Mixer usedSample replay usedTraining–inference consistency mechanisms usedAsynchronous group-wise grading optionalDeficit-corrected scheduling evaluated—Agentic reinforcement learning coreContainer-level network isolation usedEnvironment hardening usedEnvironment preparation to prevent solution leakage usedMulti-harness training usedUnified trajectory representation for agentic reinforcement learning used—Agent-centric mid-training usedMid-training alignment data used—
DeepSeek-V4-Flash core——Group Relative Policy Optimization used—Direct RL optimization of a generative reward model usedGenerative Reward Model used——Multi-Teacher On-Policy Distillation coreOn-Policy Distillation used————
MiMo-V2.5 core—Cold-started SFT model usedSupervised fine-tuning used—Group Relative Policy Optimization usedReinforcement Learning usedRollout Routing Replay used—Outcome Reward Model usedRule-based verifier usedTest-difficulty-driven code reward used——Multi-Teacher On-Policy Distillation coreOn-Policy Distillation used—Continuous rollout with asynchronous reward computation and early termination usedData re-sampling strategy usedData Scheduler usedPartial rollout usedSeamless Rollout Engine usedStaleness-aware truncated importance sampling usedTool Manager usedToolbox used—Agentic post-training usedAgentic reinforcement learning used——
GLM-5.3 usedExtended post-training used——SAO with Compaction used—————Environment scaling for realistic training tasks used——
Hy3 usedJoint SFT and RL used—Supervised fine-tuning usedFine-tuning optional—Group Relative Policy Optimization usedReinforcement Learning usedReinforcement Learning Post-Training used———————
GLM-5.2 used—Rejection sampling used—Direct Double-Sided Importance Sampling usedGroup Relative Policy Optimization usedGroup-Wise Policy Optimization usedIcePop used—Hybrid reward system usedMulti-level reward formulation used——On-Policy Cross-Stage Distillation used—Asynchronous Agent RL algorithms usedAsynchronous reinforcement learning usedAsynchronous reinforcement learning infrastructure usedMulti-Task Rollout Orchestrator usedSLIME usedToken-in-token-out (TITO) used———
MiniMax-M3 used———————Multi-turn collaboration simulator used——
DeepSeek-V3.2 coreScalable reinforcement learning post-training core—Prompt-based cold start for tool use usedReasoning-data system prompt used—Group Relative Policy Optimization coreKeep Routing coreMixed Reinforcement Learning coreKeep Sampling Mask usedOff-Policy Sequence Masking usedReinforcement Learning Training usedUnbiased KL Estimate used—Generative Reward Model usedLanguage consistency reward usedLength-adjusted RL reward usedRule-based outcome reward used——Specialist Distillation used——Generate-verify-refine loop usedReinforcement learning on synthetic agentic data usedTraining agentic policies with real-world tools usedReinforcement learning restricted to search and code environments evaluated—Continual pretraining used—
DeepSeek-V4-Flash-Vision-Exp corePost-training optimization used————————Continued training for visual understanding core—
DeepSeek-V4-Pro core—Specialist-model training usedSupervised fine-tuning used—Group Relative Policy Optimization usedReinforcement Learning used—Direct RL optimization of a generative reward model usedGenerative Reward Model used——Multi-Teacher On-Policy Distillation coreOn-Policy Distillation core—Preemptible rollout service used——Mid-training phase used—
DeepSeek-V4-Pro-0813 usedPost-training optimization used—————————
Gemma 4 31B mentioned—Fine-tuning mentioned————————
Inkling corePost-training on diverse domains used—LoRA fine-tuning coreSafety training to an internal specification usedSFT bootstrapping with synthetic data used—Large-Scale Reinforcement Learning usedReinforcement Learning for Calibration with Proper Scoring Rules usedRL with Rubric and Claims Graders used—Abstention-aware reward for factual QA used——Distillation for Post-Training Data Generation used—Large-scale asynchronous RL core———
Kimi K3 coreThree-stage post-training with multi-teacher on-policy distillation core—Supervised fine-tuning used—Multi-Domain, Multi-Effort Reinforcement Learning coreDomain-Specialized RL Experts used—Agentic Generative Reward Model usedPublic and hidden verifier pairing usedWeb-development reward with deterministic checks and model judging used—Budget-based verbosity control used—Multi-Teacher On-Policy Distillation corePer-Token On-Policy Distillation Reward used—Co-located long-context agentic RL system usedCo-located RL training usedPartial rollout used—Training on verifiable problems in agentic environments coreAutonomous Execution Tasks (AET) usedMock applications for personal-assistant reinforcement learning usedUnified white-box reinforcement-learning environment used——
Laguna-S-2.1 coreReinforcement learning for low-pass-rate tasks core—Evaluation-based early stopping usedFrozen-network warmup for new special tokens usedInstruction-following fine-tuning usedSFT bootstrapping with synthetic data used—CISPO with Length-Weighted Leave-One-Out Group-Relative Advantages coreAgentic RL Task Mix usedPer-Token Tool-Error Step Penalty usedReinforcement Learning from Code Execution Feedback usedMoonlight Scaling optional—Binary task verifier coreBinary terminal-verifier reward coreDeterministic chain of checkers corePer-token tool-error reward shaping used———Blocking in-flight rollout steps on weight updates usedCapping trajectory staleness usedClosed-loop multi-turn rollout usedExact rollout-to-production chat-template alignment assertion usedLong-horizon rollout budgets usedMulti-harness rollouts usedRL sandboxing with selective network blocking and artifact caching usedToken-in-token-out (TITO) used—Production-harness-matched reinforcement learning coreReinforcement learning in an agent harness coreMulti-harness trajectory training with native behavior preservation usedSeed-based terminal task generation used—Continual pretraining for long-context extension used—
MiMo-V2.5-Pro core—Cold-started SFT model usedSupervised fine-tuning used—Reinforcement Learning used—Rule-based verifier usedTest-difficulty-driven code reward used——Multi-Teacher On-Policy Distillation core—Continuous rollout with asynchronous reward computation and early termination usedData re-sampling strategy usedSeamless Rollout Engine used—Agentic post-training usedAgentic reinforcement learning used——
MiMo-V2.6-Pro core—SFT checkpoint for RL research coreSelf-correction cold start usedSFT on MiMo-generated task data used—Penalty Module coreDropping All-Zero-Advantage Groups defaultAdvantage Shaping usedAsynchronous Group Relative Policy Optimization usedDomain-Specific GRPO Training usedDynamic Sampling usedFlagged-Token Masking and Penalty usedFreezing the MoE Router During Reinforcement Learning usedGroup-Relative Length Penalty usedGroupwise Advantage Redistribution usedHierarchical Penalty Escalation usedKeep Candidate-Set Replay usedQuality-Weighted Advantage Redistribution usedRollout Routing Replay usedSingle Mixed Reinforcement-Learning Run usedSingle-Harness Reinforcement Learning usedZero-Reward Correction for Leaked or External Answers used—Groupwise Agentic Grading coreAdversarial screening usedBehavior rubrics usedFive-dimension comparative grading of passing patches usedGroupwise Reward Synthesis usedHack-agent screening usedMonitoring-only penalty strategy usedMultiplicative reward synthesis usedNegative checks for unintended side effects usedRule-based or model-judged trajectory error detection usedSegment-level behavioral penalties usedSolution rubrics usedTraining-time trajectory auditing usedVerifier cross-checking used——Autonomous Student Rollouts usedDistillation Fine-Tuning on MiMo-Generated Data usedModel Distillation usedMulti-Prefix Multi-Teacher On-Policy Distillation usedPrefix-Conditioned On-Policy Distillation usedTeacher-Trajectory and SFT-History Reuse used—Agent Loop coreAgent-centric rollout execution coreAsynchronous reinforcement learning coreHarness Pool coreHierarchical trajectory data organization corePayload Porter corePredictive Rollout Dispatch coreAdaptive Rollout Concurrency usedAdaptive Rollout Scheduling usedComposite early-stop strategy usedDecoupled control plane and data plane usedDialogue prefix matching usedHeterogeneous Agent Harnesses usedIncremental multimodal-delta transfer usedIsolated resettable rollout sandboxes usedMixed-task rollout infrastructure usedMixing tasks and harnesses in a single RL batch usedMultimodal rollout data handling usedOffloading blocking environment and tokenization work to background threads usedPartial rollout usedPer-source oversampling allocation usedPer-source priors for rollout sequence-length estimation usedPersistent host-actor pools usedSample Mixer usedSample replay usedTraining–inference consistency mechanisms usedAsynchronous group-wise grading optionalDeficit-corrected scheduling evaluated—Agentic reinforcement learning coreContainer-level network isolation usedEnvironment hardening usedEnvironment preparation to prevent solution leakage usedMulti-agent task decomposition and coordination usedMulti-harness training usedUnified trajectory representation for agentic reinforcement learning used—Agent-centric mid-training usedMid-training alignment data used—
NVIDIA-Nemotron-3.5-Lightning-30B-A3B core—LoRA fine-tuning usedSupervised fine-tuning usedSupervised fine-tuning at 256k sequence length usedFine-tuning with demographically balanced datasets mentioned—Group Relative Policy Optimization usedGRPO with Masked Importance Sampling used—Reinforcement learning with verifiable rewards used———Asynchronous reinforcement learning used—Multi-environment reinforcement learning coreHarness-optimized training used—Continual pretraining for long-context extension used—
Qwen3.5-397B-A17B coreScalable RL at Agent Scale coreThree-stage Thinker post-training strategy used—Lightweight speaker fine-tuning used—GSPO usedInteraction-Aligned Reinforcement Learning usedProgressively Complex Task Distributions used——Direct Preference Optimization used—On-Policy Distillation usedSpecialist Distillation used—Asynchronous RL frameworks for large-scale agent scaffolds and environment orchestration used——Continual pretraining used—
Qwen3.6-35B-A3B coreScalable RL at Agent Scale core—————————
gpt-oss-120b core—Adversarial fine-tuning usedInstruction hierarchy training usedFine-tuning optional—Chain-of-Thought Reinforcement Learning coreHelpful-Only Adversarial Reinforcement Learning usedIncremental Reinforcement Learning used——Deliberative alignment used———Agentic tool-use training usedPython tool use in chain-of-thought used——