Open Models Almanac
Overview
Learn
Models
Methods
About
Methods
single mention · not yet filed
multimodal training from step zero
The sources on this page only mention it in passing.
Explainer · D4 · 25 min
Multimodal training data: when pixels and sound enter the recipe
Images from the first step train both modalities together, at a cost in text tokens; images added later keep a finished text model, which then has to be protected