single source · other
Model soup
Model Soup is a model merging technique that performs weighted averaging of corresponding parameters across multiple models.1 It merges models by computing merged parameters using given models and normalized weights.2
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
The simplest approach performs weighted averaging of corresponding parameters across models (Wortsman et al., 2022
— LFM2 Technical ReportGiven models with parameters and normalized weights (where ), the merged parameters are computed as
— LFM2 Technical Report
Also called Model soup merging.
Filed under other