single source · optimization
Weight averaging of runs trained on different data mixes
Combines weights from separately trained runs using different data mixes by averaging them into one model.
The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See training runtime for the methods filed beside it.
Also called souping.
Filed under optimization › training runtime