Open Models Almanac
Methodsother

single source · other

Task arithmetic

Task arithmetic merges models by working with parameter deltas relative to a base model.1 It first computes task vectors representing the adaptations learned during fine-tuning, then builds the merged model from them.2

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. Task arithmetic (Ilharco et al., 2022) extends model soups by working with parameter deltas relative to a base model — LFM2 Technical Report
  2. we first compute the task vectors representing the adaptations learned during fine-tuning. The merged model is then constructed as — LFM2 Technical Report

Also called Task arithmetic model merging.

Filed under other