general family · filed under model architecture
Linear attention
A general family of attention alternatives distinguished from softmax attention, with no particular mechanism specified in the evidence.
Also called linear attention mechanism.
- sources
- 2
- models
- 2
- lab adopt it
- 1
- strongest
- core
How sources treat it
One count per evidence span, weakest treatment to strongest.
mentioned 1core 1
Documented in
Evidence
2 spans quoted from the sources, strongest treatment first.
integrates a linear attention mechanism with a sparse mixture-of-experts model
coremodel architecturein Qwen3.5-397B-A17BQwen
Hybrid architectures replace a subset of softmax attention layers with efficient alternatives such as linear attention
mentionedother
Filed alongside
Other methods under model architecture :: token mixer :: linear attention & state space.