specific method · not yet filed
Global attention with a dense feed-forward network in the first Transformer block
Also called global attention with a dense FFN.
- source
- 1
- model
- 1
- lab adopt it
- 1
- strongest
- core
How sources treat it
One count per evidence span, weakest treatment to strongest.
core 1
Documented in
Evidence
1 span quoted from the sources, strongest treatment first.
The first Transformer block uses global attention with a dense FFN.
coremodel architecturein MiMo-V2.6-Pro-RLXiaomi