single mention · model architecture
Vision-language model backbone
A vision-language model backbone encodes multimodal inputs such as images, language instructions, and sensorimotor states into contextual features.1 It combines a vision encoder with a language decoder, and its decoder layers process concatenated image, language, and state tokens before passing the features downstream.2
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
The VLM encodes these into contextual features, which condition the action expert to generate a continuous sequence of actions
— SmolVLA: Efficient Vision-Language-Action Model trained on Lerobot Community DataThe decoder layers process concatenated image, language, and state tokens. The resulting features are then passed to the action expert
— SmolVLA: Efficient Vision-Language-Action Model trained on Lerobot Community Data
Also called Vision-Language Model (VLM).
Filed under model architecture › multimodal architecture