Open Models Almanac
Methodsmodel architecturetoken mixerlinear attention & state space

general family · filed under model architecture

Linear attention

A general attention family used as an efficient alternative to softmax attention, including in hybrid architectures.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See linear attention & state space for the methods filed beside it.

Also called linear attention mechanism.

sources
2
models
2
lab adopt it
1
strongest
core

Documented in

Alibaba (Qwen)
MiniMax

Evidence

2 quotes from 2 sources, one card per model, each opening with its strongest.

How sources treat it
mentioned 1core 1

MiniMax-M3

MiniMaxdiscussed1 source

Hybrid architectures replace a subset of softmax attention layers with efficient alternatives such as linear attention

mentionedother
↑