OpenVLA / OpenVLA-OFT
OpenVLA project (Stanford et al.)We follow best-practices from LLM serving by saving and loading OpenVLA in bfloat16 precision for inference (our default approach)
defaultinference servingin OpenVLAOpenVLA team
specific method · filed under inference & serving
Inference performed with weights and activations in BF16, including an implementation that upcasts weights before execution.
The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See inference quantization for the methods filed beside it.
Also called bfloat16 inference.
2 quotes from 2 sources, one card per model, each opening with its strongest.
We follow best-practices from LLM serving by saving and loading OpenVLA in bfloat16 precision for inference (our default approach)
In this implementation, we upcast all weights to BF16 and run the model in BF16.