Open Models Almanac
Methodsmodel architecturetoken mixersoftmax attentionglobal attention

general family · filed under model architecture

Full attention

Attention over the full context, contrasted with restricted or hybrid attention patterns.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See global attention for the methods filed beside it.

sources
2
models
2
lab adopt it
1
strongest
used

Documented in

Xiaomi
MiniMax

Evidence

2 quotes from 2 sources, one card per model, each opening with its strongest.

How sources treat it
evaluated 1used 1
↑