Open Models Almanac
Methodspost-trainingsupervised fine-tuning

single source · post-training

Freeze the vision encoder during fine-tuning

Freezing the vision encoder is a fine-tuning approach that keeps the vision encoder's weights fixed while training.1 It reduces the number of parameters updated during fine-tuning, but adapting the visual features to the target scene is lost.2

Trade-off It saves training parameters and memory, but in the reported experiments it led to poor task performance.3

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. frozen vision freezes the vision encoder but fine-tunes all other weights — OpenVLA: An Open-Source Vision-Language-Action Model
  2. only fine-tuning the network’s last layer or freezing the vision encoder leads to poor performance, suggesting that further adaptation of the visual features to the target scene is crucial — OpenVLA: An Open-Source Vision-Language-Action Model
  3. freezing the vision encoder leads to poor performance — OpenVLA: An Open-Source Vision-Language-Action Model

Also called frozen vision, freezing the vision encoder.

Filed under post-training › supervised fine-tuning