SmolVLA
Hugging Facewe employ an interleaved approach, where each block contains either a CA or a SA layer.
3 more from this model
Inside the action expert, attention layers alternate between:
Interleaving self-attention and cross-attention blocks
our proposed interleaved SA+CA setup