Open Models Almanac
ModelsSmolVLA

Hugging Face

SmolVLA

What SmolVLA’s own documents say it is built from — every method with the sentence that describes it.

Parameters
0.45B
Attention
SmolVLM2-500M backbone, flow-matching action expert
Input
text+image -> action
Licence
Apache-2.0
Released
2025-06-03

Against the consensus recipe

Its documents state 0 of the 18 methods the field agrees on.

The recipe from the overview: the methods the most labs adopt, per stage. A missing one is something its documents do not say, not something the model lacks.
  1. Data curation

  2. Training objective

  3. Model architecture

  4. Optimization

  5. Post-training

  6. Inference & serving

filed at the root 1

Flow-matching action expert core

“SmolVLA is a lightweight VLA composed of a compact pretrained VLM, and an action expert trained with flow matching.”

token mixer 4

Cross-attention core

“Cross-attention (CA), where action tokens attend to the VLM’s features”

softmax attention 2

Causal self-attention core

“Self-attention (SA), where action tokens attend to each other (causally—only to the past)”

Causal attention used

“We employ a causal attention mask for the SA layers, ensuring that each action token can only attend to past tokens within the chunk”

hybrid layer stacking 1

Interleaved Self-Attention and Cross-Attention core

“we employ an interleaved approach, where each block contains either a CA or a SA layer.”

channel mixer 1

mixture of experts 1

Action expert core

“SmolVLA’s action expert is a compact transformer (~100M parameters) that generates action chunks”

multimodal architecture 7

Reduced action-expert hidden size core

“reduce its hidden size to 75% of the VLM’s, keeping the model lightweight for deployment.”

Small pretrained VLM backbone core

“Leveraging smaller pretrained VLMs”

Visual token reduction core

“Using fewer visual tokens”

Intermediate VLM-layer features core

“we use features from intermediate layers.”

Vision-language model backbone core

“We use SmolVLM2 as our VLM backbone.”

Global-image-only runtime input default

“SmolVLA uses only the global image at runtime to keep inference lightweight and fast.”

Intermediate-layer-limited action-expert conditioning default

“the action expert only attends to VLM features up to a configurable layer NN during training, set to half the total layers.”

3 more mentioned or evaluated
Bidirectional attention evaluated

softmax attention “bidirectional interaction harms performance.”

Cross-attention-only action expert evaluated

token mixer “no interaction between action tokens (pure CA)”

Sensorimotor-state projection into VLM token space evaluated

multimodal architecture “feeding the sensorimotor states to the VLM (by projecting them into token space)”

filed at the root 1

Gradient checkpointing optional

“- -policy.gradient_checkpointing=true”

training precision 1

BF16 mixed-precision training used

“we leverage bfloat16 precision and torch.compile()”

data sourcing 2

Pre-training on community-contributed robotics datasets core

“It is pretrained on a curated mix of publicly available, community-contributed datasets designed to reflect real-world variation.”

Community-driven data used

“matching the performance of much larger VLAs through community-driven data”

data filtering 5

Manual camera-viewpoint normalization used

“we manually mapped each camera to a standardized view type—prioritizing top, wrist, and side perspectives–and renamed them as OBS_IMAGE_1…”

Community robotics dataset filtering by metadata and quality used

“we selected a subset of 481 community datasets obtained from Hugging Face, filtered according to embodiment type, episode count, overall…”

Joint-space similarity filtering of observations used

“observations are compared in joint-space, and near-duplicates are dropped.”

Manual review of candidate robotics datasets used

“After a meticulous manual review (special thanks to Marina Barannikov), we curated a collection of 487 high-quality datasets”

Robotics dataset selection by frame count, visual quality, and task coverage used

“We used a customfiltering tool created by Alexandre Chapin and Ville Kuosmanen to select datasets based on frame count, visual quality…”

synthetic data 3

Standardized action-oriented task descriptions from frames and labels used

“To improve quality and standardize the textual input across datasets, we used Qwen2.5-VL-3B-Instruct to generate concise, action-oriented…”

Frame-and-instruction prompting for automatic task annotation used

“For each dataset, we sampled representative frames and provided them alongside the original instruction.”

Rewrite task instructions under 30 characters beginning with an action verb used

“the model was prompted to rewrite the instruction in under 30 characters, starting with an action verb”

data mixture & curriculum 1

Multi-task training used

“Unless specified otherwise, SmolVLA is always trained in a multi-task setting.”

sequence packing 1

Fixed-shape batching with excess-frame truncation used

“we maintain a fixed sequence length and batch size, discarding any excess frames in an episode that do not fit a complete batch.”

filed at the root 2

Task-specific post-training core

“SmolVLA goes through a pretraining phase on general manipulation data, followed by task-specific post-training.”

Pretraining on general manipulation data core

“SmolVLA goes through a pretraining phase on general manipulation data, followed by task-specific post-training.”

supervised fine-tuning 4

Fine-tuning used

“Base model to fine tune on your specific use case”

Imitation learning used

“Our current approach primarily relies on imitation learning.”

Multitask fine-tuning used

“Multitask finetuning further boosts performance, showing strong task transfer capabilities even in low-data regimes.”

Frozen-backbone action-expert fine-tuning used

“We train only the action expert module, keeping the VLM frozen.”

1 more mentioned or evaluated
Asynchronous reinforcement learning evaluated

rollout & RL infrastructure “the async mode decouples action execution and policy inference, allowing predictions and control to run in parallel.”

filed at the root 1

Chunk fusion core

“Overlapping actions from successive chunks are stitched with a simple merge rule to avoid jitter.”

inference scheduling 2

Asynchronous inference and execution core

“we propose an asynchronous inference stack that enables faster adaptation and responsiveness in real-world manipulation tasks.”

Threshold-triggered inference core

“When the queue length falls below a threshold (e.g., 70%), we send an observation to a Policy Server”

1 more mentioned or evaluated
Action-chunk execution evaluated

decoding strategy “the robot can execute multiple actions from the predicted chunk before processing new observations”

unfiled 21

Flow matching core

“Training objective: flow matching”

Chunked action prediction core

“Our model predicts chunks of actions”

Flow matching objective for continuous action prediction core

“It is trained using a flow matching objective, which teaches the model to guide noisy samples back to the ground truth.”

Noise action sequences and predict a correction vector core

“during training, we add random noise to the robot’s real action sequences and ask the model to predict the “correction vector””

Skip upper vision-language model layers for inference efficiency core

“Skipping half of the layers of the vision model for faster inference and smaller size”

Cap visual tokens at 64 per frame default

“SmolVLA limits the number of visual tokens to 64 per frame during both training and inference.”

Action chunk sizes of 10–50 time steps used

“chunk sizes between 10 and 50 provide a good balance between the robot reactivity and efficiency.”

Dropping unused camera views during training used

“unused views were dropped during training.”

Overlapping action-queue aggregation used

“triggering chunk prediction while the control loop is still consuming a previously available queue, aggregating it with the newly incoming…”

Per-action observation and prediction in simulation used

“In simulation, we perform inference by sampling new observations and predicting a new action after each executed action.”

Pixel-shuffle visual token reduction used

“in addition to a pixel shuffle operation, limiting the visual tokens to 64 per frame.”

PixelShuffle image compression for visual-token reduction used

“using PixelShuffle as an efficient shuffling technique.”

PyTorch JIT compilation with torch.compile used

“torch.compile() that JIT-compiles PyTorch code into optimized kernels.”

Queue-threshold-triggered observation capture used

“consuming actions from a readily available queue until a threshold condition on the number of remaining actions in the queue is met.”

Synchronous action-chunk inference used

“For real-world evaluation, we perform synchronous inference: the model samples new observations only after executing the full chunk of…”

VLM layer skipping used

“Key design choices include: (i) skipping layers in the VLM”

5 more mentioned or evaluated
Frequent observation updates during chunk execution evaluated

“updating observations more frequently significantly improves success rate”

L1 regression loss on predicted action chunks evaluated

“a standard regression L1 loss on the predicted versus ground-truth action chunks.”

Pass sensorimotor states directly to the action expert evaluated

“passing them directly to the action expert.”

Skip every second VLM layer evaluated

“we also test a variant sampling every second VLM layer (Skip % 2)”

Using only early LLM layers of the VLM evaluated

“using only the first half of the VLM layers gives a good trade-off between performance and compute.”

Sources

The curated source list for this model. "not fetched" means the URL is recorded but its text was not read in the current run.

SourceKindPublisherStatus
huggingface.co/lerobot/smolvla_basemodel cardHugging Faceread
arxiv.org/abs/2506.01844technical reportHugging Faceread
huggingface.co/blog/smolvlaofficial blogHugging Faceread
↑