Open Models Almanac
Methodsmodel architecturemultimodal architecture

single mention · model architecture

Data-parallel-first multimodal encoding

Balances image encoding across ranks before redistributing embeddings to align with sequence tokens.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See multimodal architecture for the methods filed beside it.

Also called encoding runs data-parallel first.

Filed under model architecture › multimodal architecture