Open Models Almanac
Methodspost-trainingsupervised fine-tuning

single mention · post-training

Specialist training

Specialist training fine-tunes separate models on domain-specific curated data, so each specialist learns one domain deeply.1 These specialists' behaviour is then transferred to the final model through full-vocabulary distillation, which preserves both correct and incorrect signals.2

Trade-off Because distillation faithfully preserves wrong signals too, specialist training data needs a higher quality bar than general-purpose fine-tuning.3

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. A math specialist needs proofs verified by mathematicians. A coding specialist needs solutions tested against real compilers and test suites — DeepSeek V4 Guide: Engram Memory, Training Data Strategy & Release Status (2026)
  2. a wrong logit distribution transmitted through distillation to the final model — DeepSeek V4 Guide: Engram Memory, Training Data Strategy & Release Status (2026)
  3. The quality bar for specialist training data is higher than for general-purpose fine-tuning — DeepSeek V4 Guide: Engram Memory, Training Data Strategy & Release Status (2026)

Also called independent specialist models.

Filed under post-training › supervised fine-tuning