Open Models Almanac
Methodsmodel architecturemultimodal architecture

specific method · filed under model architecture

Early-fusion multimodal training

Qwen's term for training on multimodal tokens; its cards do not give the step at which images enter. The Qwen3.5 models still have a separate vision encoder and an MLP projector, so the term is about when vision is trained, not about removing the encoder.

Written for the explainer Getting images and video into a decoder: encoders, projectors, and native fusion.

Also called Early fusion training on multimodal tokens, Unified Vision-Language Foundation, Early fusion vision-language training.

sources
5
models
2
lab adopt it
1
strongest
core

Documented in

Evidence

5 quotes from 5 sources, one card per model, each opening with its strongest.

How sources treat it
used 1core 4

Qwen3.5-397B-A17B

Alibaba (Qwen)adopted4 sources

Early fusion training on multimodal tokens achieves cross-generational parity with Qwen3 and outperforms Qwen3-VL models across reasoning, coding, agents, and visual understanding benchmarks.

coremodel architecturein Qwen3.5-122B-A10BQwen Team
3 more from this model

Early fusion training on multimodal tokens achieves cross-generational parity with Qwen3 and outperforms Qwen3-VL models across reasoning, coding, agents, and visual understanding benchmarks.

coremodel architecturein Qwen3.5-35B-A3BQwen Team

Early fusion training on multimodal tokens achieves cross-generational parity with Qwen3

usedtraining objectivein Qwen3.5-397B-A17BQwen Team

Early fusion training on multimodal tokens means the model doesn’t need a separate vision adapter.

coremodel architecturein Qwen3.5-397B-A17BAlibaba

Qwen3.6-35B-A3B

Alibaba (Qwen)adopted1 source

Early fusion training on trillions of multimodal tokens achieves cross-generational parity with Qwen3 and outperforms Qwen3-VL models across reasoning, coding, agents, and visual understanding benchmarks.

coremodel architecturein Qwen3.5Qwen Team

Further reading

Picked by hand, not extracted: where to read more, not evidence for anything on this page.

↑