general family · filed under model architecture
Full attention
Attention over the full context, contrasted with restricted attention such as sliding-window attention.
- sources
- 2
- models
- 2
- lab adopt it
- 1
- strongest
- used
How sources treat it
One count per evidence span, weakest treatment to strongest.
evaluated 1used 1
Documented in
Evidence
2 spans quoted from the sources, strongest treatment first.
full-attention
usedmodel architecturein MiMo-V2.5Xiaomi
MSA matched full attention on the vast majority of capabilities
evaluatedevaluation onlyin MiniMax-M3MiniMax
Filed alongside
Other methods under model architecture :: token mixer :: softmax attention :: global attention.