Open Models Almanac
Methodsoptimizationtraining runtime

single source · optimization

Weight averaging of runs trained on different data mixes

Combines weights from separately trained runs using different data mixes by averaging them into one model.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See training runtime for the methods filed beside it.

Also called souping.

Filed under optimization › training runtime