Open Models Almanac
Methodsmodel architecturetoken mixersoftmax attentionsliding window attention

single mention · model architecture

Alternating Row-Major and Column-Major Token Serialization

Alternates the token serialization order between local SWA layers to propagate information along both spatial axes.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See sliding window attention for the methods filed beside it.

Filed under model architecture › token mixer › softmax attention › sliding window attention