Open Models Almanac
Methodsother

single source · other

Model soup

Model Soup is a model merging technique that performs weighted averaging of corresponding parameters across multiple models.1 It merges models by computing merged parameters using given models and normalized weights.2

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. The simplest approach performs weighted averaging of corresponding parameters across models (Wortsman et al., 2022 — LFM2 Technical Report
  2. Given models with parameters and normalized weights (where ), the merged parameters are computed as — LFM2 Technical Report

Also called Model soup merging.

Filed under other