Open Models Almanac
Methodsmodel architecturetoken mixersparse attentionblock selection

methods · sparse attention

block selection

9 methods filed at this node or below it, from the sources of 3 models.

In this branch 9

Everything filed at this node or below it, with one collapsible heading per child node.

filed here 9

MiniMax Sparse Attention core · 7 sources · 8 quotes
Top-k block selection used · 2 sources · 2 quotes
7 single mentions
Block-level selection core · 1 source · 1 quote
Dynamic sparse selection core · 1 source · 1 quote
Main Branch core · 1 source · 2 quotes
Top-512 block selection core · 1 source · 1 quote
Block Max Pooling used · 1 source · 1 quote
Top-scoring compressed-block selection used · 1 source · 1 quote
Mixture of Block Attention evaluated · 1 source · 1 quote

By model

Which of this branch's methods each model's own documents describe, and how strongly. Under each model: its strongest treatment anywhere in the branch.

↑