OpenVLA project (Stanford et al.)
OpenVLA / OpenVLA-OFT
What OpenVLA / OpenVLA-OFT’s own documents say it is built from — every method with the sentence that describes it.
- Parameters
- 7.5B
- Attention
- Llama 2 7B + fused DINOv2/SigLIP encoder; 256-bin action tokens; OFT: parallel decoding, action chunks, continuous L1 head
- Input
- text+image -> action
- Licence
- MIT code; Llama 2 Community License weights
- Released
- 2024-06-10
Against the consensus recipe
Its documents state 0 of the 18 methods the field agrees on.
Data curation
- Benchmark decontaminationnot in its documents
- Synthetic data generationnot in its documents
- Corpus deduplicationnot in its documents
Training objective
- Multi-Token Predictionnot in its documents
- Knowledge insulationnot in its documents
- Quantization-aware distillationnot in its documents
Model architecture
- Mixture of expertsnot in its documents
- Multi-Token Predictionnot in its documents
- Grouped-query attentionnot in its documents
Optimization
- Muonnot in its documents
- Expert parallelismnot in its documents
- Quantization-Aware Trainingnot in its documents
Post-training
- Supervised fine-tuningnot in its documents
- Asynchronous reinforcement learningnot in its documents
- Multi-Teacher On-Policy Distillationnot in its documents
Inference & serving
- Thinking modenot in its documents
- Multi-Token Predictionnot in its documents
- FP8not in its documents
Checked in its code
Its code shows 0 of the 19 architecture features checked.
Sources say what a lab chose to describe; the modeling code says what the checkpoint runs. Each feature below was put to a code verifier, and only the config values and source lines it cited that were found in the files are shown, linked to the line.
Read from the model repository’s own code (modeling_prismatic.py, configuration_prismatic.py) and config.json at revision 47a0ec7fc4, by anthropic/claude-sonnet-5 on 2026-11-07.
| Feature | Code | Its documents | Verdict |
|---|---|---|---|
| Grouped-query attention | code unclear | not stated | The language model is loaded via AutoModelForCausalLM.from_config using the Llama-2-7b text_config, but the actual Llama attention… show evidenceThe language model is loaded via AutoModelForCausalLM.from_config using the Llama-2-7b text_config, but the actual Llama attention implementation (and any num_key_value_heads setting) is not present in the given files or config, so GQA cannot be confirmed from what is provided. text_config.model_type = "llama" |
| Multi-head latent attention | not in its code | not stated | No kv_lora_rank, q_lora_rank, or latent attention mechanism appears anywhere in the config or code; the language model is a stock Llama-2… show evidenceNo kv_lora_rank, q_lora_rank, or latent attention mechanism appears anywhere in the config or code; the language model is a stock Llama-2 backbone. text_config.model_type = "llama" |
| Sliding-window attention | not in its code | not stated | There is no sliding_window, use_sliding_window, or layer_types key in text_config, and no windowed-attention code is present in the given… show evidenceThere is no sliding_window, use_sliding_window, or layer_types key in text_config, and no windowed-attention code is present in the given files. text_config.model_type = "llama" |
| Interleaved sliding-window and global attention | not in its code | not stated | No layer_types or sliding_window_pattern configuration exists, and no interleaving code is present; attention layers are uniform. show evidencetext_config.model_type = "llama" |
| Indexer-selected sparse attention (DeepSeek Sparse Attention) | not in its code | not stated | No indexer-related config keys (index_topk, index_n_heads, index_head_dim) or code implementing a top-k token selector are present. |
| Linear-attention or state-space layers alongside full attention | not in its code | not stated | No linear-attention/state-space layer config (layer_types, hybrid_override_pattern, linear_attn_config) or code is present; the model uses… show evidenceNo linear-attention/state-space layer config (layer_types, hybrid_override_pattern, linear_attn_config) or code is present; the model uses a standard Llama transformer. text_config.model_type = "llama" |
| Gated DeltaNet layers | not in its code | not stated | No gated delta-rule or Kimi Delta Attention mechanism or associated config keys (linear_num_key_heads, linear_conv_kernel_dim) appear in… show the whole verdictNo gated delta-rule or Kimi Delta Attention mechanism or associated config keys (linear_num_key_heads, linear_conv_kernel_dim) appear in the files or config. |
| Mamba-2 layers | not in its code | not stated | No Mamba-2/SSD config keys (mamba_num_heads, ssm_state_size, hybrid_override_pattern) or code implementing selective state-space blocks… show the whole verdictNo Mamba-2/SSD config keys (mamba_num_heads, ssm_state_size, hybrid_override_pattern) or code implementing selective state-space blocks are present. |
| Learnable attention sink | not in its code | not stated | No learned per-head sink logit mechanism is implemented anywhere in the provided modeling code. |
| Gated attention output | not in its code | not stated | No sigmoid gating of attention output is implemented in the provided code; the language model is instantiated from a stock Llama config… show the whole verdictNo sigmoid gating of attention output is implemented in the provided code; the language model is instantiated from a stock Llama config with no gate parameters referenced. |
| QK normalization | not in its code | not stated | No use_qk_norm, qk_layernorm config key or per-head query/key normalization code is present in the given files. |
| Partial RoPE | code unclear | not stated | The language model backbone is standard Llama-2, whose RoPE implementation is not included in these files, and no partial_rotary_factor or… show evidenceThe language model backbone is standard Llama-2, whose RoPE implementation is not included in these files, and no partial_rotary_factor or qk_rope_head_dim key appears in text_config to confirm or deny partial RoPE. text_config.model_type = "llama" |
| YaRN RoPE scaling | not in its code | not stated | There is no rope_scaling or rope_parameters key in text_config, so no YaRN scaling is configured. show evidencetext_config.model_type = "llama" |
| Layers without positional encoding (NoPE) | not in its code | not stated | No no_rope_layers, nope_layer_interval, or layer_types config exists, and no code path skips positional encoding for some attention layers. |
| Mixture of experts | not in its code | not stated | No num_experts/n_routed_experts/num_local_experts config or MoE routing code is present; the feed-forward network is the standard dense… show evidenceNo num_experts/n_routed_experts/num_local_experts config or MoE routing code is present; the feed-forward network is the standard dense Llama MLP. text_config.model_type = "llama" |
| Shared expert | not in its code | not stated | No shared-expert config keys (n_shared_experts, shared_expert_intermediate_size) or code are present, consistent with there being no MoE… show the whole verdictNo shared-expert config keys (n_shared_experts, shared_expert_intermediate_size) or code are present, consistent with there being no MoE at all. |
| Auxiliary-loss-free load balancing (selection bias) | not in its code | not stated | No topk_method, e_score_correction_bias, or router bias mechanism is present since there is no MoE router in this model. |
| Multi-token prediction layers | not in its code | not stated | No num_nextn_predict_layers, mtp_num_layers, or similar config key or extra prediction-module code is present in the given files. |
| SwiGLU feed-forward | code unclear | not stated | The language model is a stock Llama-2 backbone loaded via AutoModelForCausalLM, whose MLP activation (hidden_act) is not set in the given… show evidenceThe language model is a stock Llama-2 backbone loaded via AutoModelForCausalLM, whose MLP activation (hidden_act) is not set in the given text_config and the implementing file is not provided, so SwiGLU cannot be confirmed from this input. text_config.model_type = "llama" |
filed at the root 1
“our spatially-agnostic modulation approach more closely mirrors FiLM’s operation in convolutional networks”
token mixer 1
softmax attention 1
“We also use a bidirectional attention mask (instead of causal), enabling the model to leverage all intermediate features non-causally when…”
channel mixer 1
“we employ feature-wise linear modulation (FiLM) [37], which infuses language embeddings into the visual representations so that the model…”
prediction head 2
“we replace this input with empty action embeddings that differ only by their positional encoding values”
“For parallel decoding, we replace this input with empty action embeddings that differ only by their positional encoding values”
multimodal architecture 10
“the final hidden states are instead mapped directly to normalized continuous actions by a separate action head MLP”
“The OpenVLA model architecture consists of a fused vision backbone that combines the SigLIP [9] and DinoV2 [25] encoders.”
“processes multiple input images (e.g., third-person image plus wrist camera images) through the shared SigLIP-DINOv2 backbone”
“directly fine-tuning VLMs to generate robot actions by treating them as tokens in the language model vocabulary”
“projects robot proprioceptive state to language embedding space via a 2-layer MLP with GELU activation”
“We apply FiLM after the self-attention layer and before the feedforward layer in each vision transformer block”
“trained from multi-resolution visual features, fusing low-level spatial information from DINOv2 [25] with higher-level semantics from…”
“patch features from pretrained visual transformers are treated as tokens, and are then projected into the input space of a language model”
“All input embeddings—visual features, robot state, and language tokens—are concatenated along the sequence dimension before being passed…”
“For all methods in our comparisons, we only utilize the static third-person camera images; we do not use the wrist camera images”
1 more mentioned or evaluated
multimodal architecture “consists of only the SigLIP encoder as the vision backbone”
language modelling objective 1
“OpenVLA is trained with a standard next-token prediction objective, evaluating the cross-entropy loss on the predicted action tokens only”
filed at the root 1
“we found it important for VLA training to iterate through the training dataset significantly more times”
learning-rate schedule 2
“achieved the best results using a fixed learning rate of 2e-5”
“For faster convergence, we decay the learning rate from 5e-4 to 5e-5 after 100K gradient steps.”
training precision 1
“supports modern techniques for large transformer model training such as automatic mixed precision (AMP, PyTorch [75])”
training parallelism 1
“fully sharded data parallelism (FSDP, Zhao et al. [77])”
Not used 1
learning-rate schedule “We did not find learning rate warmup to provide benefits”
data sourcing 1
“trained on 970K robot manipulation episodes from the Open X-Embodiment dataset”
data filtering 4
“restrict our training dataset to contain only manipulation datasets with at least one 3rd person camera and use single-arm end-effector…”
“We filter out all “no-op” actions from the dataset”
“Therefore, we simply filtered out the first transition in every demonstration when training the OpenVLA model”
“we replay all demonstrations in the corresponding simulation environments and filter out the demonstrations that fail to complete the task”
data mixture & curriculum 3
“OpenVLA is trained on a large dataset of robot embodiments, scenes, and tasks from the Open X-Embodiment dataset”
“we removed DROID from the data mixture for the final third of training”
“we leverage the data mixture weights of Octo [5] for all datasets that pass the first round of filtering”
tokenization 3
“we discretize each dimension of the robot actions separately into one of 256 bins”
“we set the bin width to uniformly divide the interval between the 1st and 99th quantile of the actions in the training data”
“overwriting the 256 least used tokens in the Llama tokenizer’s vocabulary (which corresponds to the last 256 tokens) with our action tokens”
supervised fine-tuning 12
“directly fine-tuning visually-conditioned language models (VLMs) such as PaLI [19, 20] to generate robot control actions”
“we fine-tuned all 7B parameters in the model, including the SigLIP-DinoV2 vision backbone”
“our Optimized Fine-Tuning (OFT) recipe shows promise for adapting VLAs to novel robots and tasks”
“we found fine-tuning the vision encoder during VLA training to be crucial for good VLA performance”
“LoRA uses the popular low-rank adaptation technique of Hu et al. [26] with multiple rank values, applied to all linear layers of the model”
“full fine-tuning of all model parameters, using small datasets with 10–150 demonstrations of a target task”
“we fine-tune the base VLA end-to-end with techniques that enable high efficiency without needing a separate controller”
“we develop high-performing policies just by training once offline on a fixed dataset of expert task demonstrations”
“OpenVLA can be easily adapted to new robot setups via parameter-efficient fine-tuning techniques.”
“we suggest collecting a dataset of demonstrations on the desired setup, and fine-tuning OpenVLA models instead”
“we focus solely on supervised fine-tuning on the target task suite, measuring the performance of various policies trained via behavioral…”
“efficiently fine-tuned for new tasks and robot setups given minimal demonstration data”
4 more mentioned or evaluated
supervised fine-tuning “fine-tuning it solely on BridgeData V2 rather than the entire OpenX training mixture”
supervised fine-tuning “frozen vision freezes the vision encoder but fine-tunes all other weights”
supervised fine-tuning “last layer only fine-tunes only the last layer of OpenVLA’s transformer backbone and the token embedding matrix”
supervised fine-tuning “sandwich fine-tuning unfreezes the vision encoder, token embedding matrix, and last layer”
filed at the root 1
“we implement a remote VLA inference server to allow real-time remote streaming of action predictions to the robot”
decoding strategy 2
“action = vla.predict_action(**inputs, unnorm_key="bridge_orig", do_sample=False)”
“we mitigated this issue by simply querying the second-most-likely action from the model”
inference quantization 2
“We follow best-practices from LLM serving by saving and loading OpenVLA in bfloat16 precision for inference (our default approach)”
“We can further reduce the memory footprint of OpenVLA during inference via quantization, without compromising performance”
inference kernel 2
“supports modern techniques for large transformer model training such as automatic mixed precision (AMP, PyTorch [75]), FlashAttention [76]”
“"attn_implementation="flash_attention_2", # [Optional] Requires `flash_attn`”
5 more mentioned or evaluated
inference quantization “4-bit quantization results in similar performance as bfloat16 half-precision inference despite requiring less than half the amount of GPU…”
inference quantization “we investigate serving the OpenVLA model with 8-bit and 4-bit precision on 8 representative BridgeData V2 tasks”
inference quantization “both 8-bit and 4-bit quantization achieve comparable token accuracy to bfloat16 inference when evaluated offline on training data.”
inference quantization “we evaluated OpenVLA with different levels of precision at inference time: half precision (bfloat16), 8-bit quantization, and 4-bit…”
decoding strategy “Exploring the use of action chunking or alternative inference-time optimization techniques such as speculative decoding [91] offer…”
Not used 1
decoding strategy “we empirically found that dynamic querying led to worse performance than simply querying the second-most-likely action at all times”
unfiled 10
“to predict actions for multiple future timesteps, we simply insert additional empty action embeddings in the decoder’s inputs, which are…”
“The model is trained to minimize the mean L1 difference between predicted and ground-truth actions”
“Separate projectors are learned for each transformer block to allow for block-specific modulation patterns.”
“We train until the mean L1 loss between predicted and ground-truth normalized actions (scaled between ) falls below 0.01.”
“L1 regression: The MLP action head consists of 4 layers with ReLU activation, mapping final Llama-2 decoder layer hidden states directly…”
“L1 regression may help smoothen out noise in training demonstrations by encouraging the policy to learn the median mode in demonstrated…”
“actions need to be un-normalized subject to statistics computed on a per-robot, per-dataset basis”
3 more mentioned or evaluated
“each action is fully executed on the robot before the next one is predicted by the policy and executed by the controller”
“During inference, the policy gradually denoises noisy action samples via reverse diffusion to produce real actions.”
“incorporating action chunking and temporal smoothing, as implemented in Diffusion Policy”
Sources
The curated source list for this model. "not fetched" means the URL is recorded but its text was not read in the current run.
| Source | Kind | Publisher | Status |
|---|---|---|---|
| huggingface.co/openvla/openvla-7b | model card | OpenVLA | read |
| arxiv.org/abs/2406.09246 | technical report | OpenVLA | read |
| arxiv.org/abs/2502.19645 | technical report | OpenVLA | read |