Open Models Almanac
Methodsmodel architecturetoken mixersparse attentionfixed-pattern sparse attention

methods · sparse attention

fixed-pattern sparse attention

3 methods filed at this node or below it, from the sources of 2 models.

In this branch 3

Everything filed at this node or below it, with one collapsible heading per child node.

filed here 3

all mentioned once or only as not used
Sparse Global Attention Anchors core · 1 source · 1 quote
Local Block used · 1 source · 1 quote
Forced Sink and Local Window not used · 1 source · 1 quote

By model

Which of this branch's methods each model's own documents describe, and how strongly. Under each model: its strongest treatment anywhere in the branch.

↑