Open Models Almanac
Methodsmodel architecturemultimodal architecture

single mention · model architecture

Sensorimotor-state projection into VLM token space

A sensorimotor-state projection turns the robot's sensorimotor states into tokens in the VLM's embedding space.1 It lets the VLM see state information directly, instead of passing the states only to the action expert, which the authors report yields better performance.2

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. feeding the sensorimotor states to the VLM (by projecting them into token space — SmolVLA: A vision-language-action model for affordable and efficient robotics
  2. including state information in the VLM leads to significantly better performance — SmolVLA: A vision-language-action model for affordable and efficient robotics

Also called feeding the sensorimotor states to the VLM.

Filed under model architecture › multimodal architecture