Open Models Almanac
Methodsmodel architecturemultimodal architecture

single mention · model architecture

Connector pre-training with frozen encoders

Connector pre-training is the first stage of multimodal training, which aligns the visual and textual embedding spaces.1 It trains only the connector, keeping the image encoder and language backbone frozen, to map image patch embeddings into the language embedding space before full multimodal training.2

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. The first stage focuses on aligning the visual and textual embedding spaces — LFM2 Technical Report
  2. while keeping the image encoder and language backbone frozen, establishing a mapping from image patch embeddings into the language embedding space before full multimodal training — LFM2 Technical Report

Also called Connector Pre-training.

Filed under model architecture › multimodal architecture