single mention · post-training
Specialist training
Specialist training fine-tunes separate models on domain-specific curated data, so each specialist learns one domain deeply.1 These specialists' behaviour is then transferred to the final model through full-vocabulary distillation, which preserves both correct and incorrect signals.2
Trade-off Because distillation faithfully preserves wrong signals too, specialist training data needs a higher quality bar than general-purpose fine-tuning.3
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
A math specialist needs proofs verified by mathematicians. A coding specialist needs solutions tested against real compilers and test suites
— DeepSeek V4 Guide: Engram Memory, Training Data Strategy & Release Status (2026)a wrong logit distribution transmitted through distillation to the final model
— DeepSeek V4 Guide: Engram Memory, Training Data Strategy & Release Status (2026)The quality bar for specialist training data is higher than for general-purpose fine-tuning
— DeepSeek V4 Guide: Engram Memory, Training Data Strategy & Release Status (2026)
Also called independent specialist models.
Filed under post-training › supervised fine-tuning