Open Models Almanac
Methodsmodel architecturetoken mixersoftmax attentionglobal attention

methods · softmax attention

global attention

5 methods filed at this node or below it, from the sources of 7 models.

In this branch 5

Everything filed at this node or below it, with one collapsible heading per child node.

filed here 5

Global attention core · 6 sources · 6 quotes
Unified Keys and Values core · 2 sources · 2 quotes
Full attention used · 2 sources · 2 quotes
2 single mentions
Full attention without positional encoding core · 1 source · 1 quote

By model

Which of this branch's methods each model's own documents describe, and how strongly. Under each model: its strongest treatment anywhere in the branch.

↑