MolmoAct 2
Allen Institute for AIuse PyTorch’s scaled dot-product attention implementation rather than FlashAttention because our packed multimodal sequences and robot-conditioning masks require custom attention masks.
usedunclearin MolmoAct2 training implementationAllen Institute for Artificial Intelligence (AI2)
1 more from this model
use PyTorch’s Scaled Dot-Product Attention (SDPA) attention implementation.
usedsoftware implementationin MolmoActAllen Institute for Artificial Intelligence (AI2)