Model techniques map
Taxonomyoptimizationtraining stability

taxonomy node · level 2

training stability

17 methods filed at this node or below it, from the sources of 11 models.

optimization :: training stability

Matching aids for the classifier: gradient clipping; soft dropping; weight padding; anti-hallucination training.

In this branch 17

Everything filed at this node or below it, with one collapsible heading per child node.

filed here 17

Weight padding used · 2 sources · 2 quotes
Anti-hallucination training used · 1 source · 1 quote
Behavioral regularization used · 1 source · 1 quote
Cross-replica model-weight hash consistency checks used · 1 source · 1 quote
Elevated constant-learning-rate training stability stress test used · 1 source · 1 quote
Exponential moving average of checkpoints used · 1 source · 1 quote
Gradient clipping used · 1 source · 1 quote
Learning-rate elevation for training stability stress testing used · 1 source · 1 quote
Loss masking for excessively stale tokens used · 1 source · 1 quote
Off-policy sample filtering used · 1 source · 1 quote
Per-token regularization for off-policy RL used · 1 source · 1 quote
Safeguards against training drift and reward hacking used · 1 source · 1 quote
Soft dropping used · 1 source · 1 quote
Training stability stress testing used · 1 source · 1 quote
Weight clipping used · 1 source · 1 quote
Weight decay coupled to learning-rate squared used · 1 source · 1 quote
Activation or logit clipping not used · 1 source · 1 quote

By model

Which of this branch's techniques each model's own documents describe, and how strongly. Under each model: its strongest treatment anywhere in the branch.

Modeltechniques
DeepSeek-V4.1-Flash usedLoss masking for excessively stale tokens used—
NVIDIA-Nemotron-3-Ultra-550B-A55B usedWeight padding used—
MiMo-V2.6-Flash usedBehavioral regularization usedSafeguards against training drift and reward hacking used—
MiMo-V2.5 usedGradient clipping used—
Hy3 usedAnti-hallucination training used—
GLM-5.2 usedOff-policy sample filtering used—
Inkling usedWeight decay coupled to learning-rate squared used—
Kimi K3 usedPer-token regularization for off-policy RL usedSoft dropping usedWeight clipping used—
Laguna-S-2.1 usedCross-replica model-weight hash consistency checks usedExponential moving average of checkpoints used—
MiMo-V2.6-Pro usedBehavioral regularization usedSafeguards against training drift and reward hacking used—
Qwen3.8-Flash-Next usedElevated constant-learning-rate training stability stress test usedLearning-rate elevation for training stability stress testing usedTraining stability stress testing usedActivation or logit clipping not used—