specific method · filed under model architecture
Mamba-2
The Mamba-2 state-space model, named separately from Mamba and used for sequence processing.
Also called Mamba-2 state-space model.
- sources
- 5
- models
- 2
- lab adopt it
- 1
- strongest
- core
How sources treat it
One count per evidence span, weakest treatment to strongest.
used 1core 5
Documented in
Further reading
Picked by hand, not extracted: where to read more, not evidence for anything on this page.
- Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality (Dao & Gu, 2024) paper arxiv.orgintroduces Mamba-2
In the code
Checked as “Mamba-2 layers” against each model’s own modeling code and config.json, independently of its documents. A model’s page shows the quoted lines.
| Model | Code | Its documents | What the code does |
|---|---|---|---|
| GLM-5.3-Flash | not in its code | not stated | No Mamba-2 SSD selective state-space block exists; the linear-attention layers implement Kimi Delta Attention instead, and no mamba_num_heads/ssm_state_size config or code is present. |
| DeepSeek-V4.1-Flash | not in its code | not stated | No Mamba-2/SSD selective state-space implementation is present in the given files. |
| Hy4-preview | not in its code | not stated | No Mamba-2/SSD state-space code or config keys (mamba_num_heads, ssm_state_size) are present. |
| DeepSeek-V4-Flash-0731 | not in its code | not stated | No Mamba-2/SSD selective state-space block is implemented; the model is built entirely from DeepseekV4Attention layers. |
| NVIDIA-Nemotron-3-Ultra-550B-A55B | in its code | core | Linear-attention layers are implemented as NemotronHMamba2Mixer, a Mamba-2 SSD block with per-head scalar decay (A_log/D), chunked scan, and mamba_num_heads=256, ssm_state_size=128. modeling_nemotron_h.py#L366 |
| MiMo-V2.6-Flash | not in its code | not stated | No Mamba-2/SSD selective state-space block is implemented anywhere in the given files. |
| DeepSeek-V4-Flash | not in its code | not stated | No Mamba-2/SSD selective state-space block is implemented anywhere in this file. |
| MiMo-V2.5 | not in its code | not stated | No Mamba-2 SSD selective state-space blocks are implemented; no mamba_num_heads or ssm_state_size config exists. |
| GLM-5.3 | not in its code | not stated | No Mamba-2/SSD selective state-space block or ssm_state_size/mamba_num_heads config is present. |
| Hy3 | not in its code | not stated | No Mamba-2/SSD selective state-space block is implemented; the config lacks mamba_num_heads or ssm_state_size |
| GLM-5.2 | not in its code | not stated | No SSD/Mamba-2 selective state-space block or associated config keys (mamba_num_heads, ssm_state_size) exist in this model. |
| MiniMax-M3 | not in its code | not stated | No Mamba-2/SSD state-space block implementation or config keys are present. |
| DeepSeek-V3.2 | not in its code | not stated | No Mamba-2/SSD state-space block or related config keys (mamba_num_heads, ssm_state_size) are present in this model. |
| DeepSeek-V4-Flash-Vision-Exp | not in its code | not stated | No Mamba-2 SSD/selective state-space blocks are implemented in this file. |
| DeepSeek-V4-Pro | not in its code | not stated | No Mamba-2 SSM blocks are implemented in the given files |
| DeepSeek-V4-Pro-0813 | not in its code | not stated | No Mamba-2 SSD/selective state-space block is implemented anywhere in this file. |
| Gemma 4 31B | not in its code | not stated | No Mamba-2/SSD state-space block implementation is present in the code. |
| Inkling | not in its code | not stated | No Mamba-2 SSD selective state-space blocks are implemented; mamba2_cache_params exists only as a stub returning a conv-only cache shape with temporal size (0,0,0), and there is no mamba_num_heads or ssm_state_size in config. configs.py#L203 |
| Kimi K3 | not in its code | not stated | No Mamba-2/SSD selective state-space block (mamba_num_heads, ssm_state_size) exists in the code or config; the linear-attention layers use KDA (delta-rule), not Mamba-2. |
| Laguna-S-2.1 | not in its code | not stated | No selective state-space (SSD/Mamba-2) block implementation is present in the modeling file. |
| MiMo-V2.5-Pro | not in its code | not stated | No Mamba-2 SSD state-space block implementation or related config keys (mamba_num_heads, ssm_state_size) are present. |
| MiMo-V2.6-Pro | not in its code | not stated | No Mamba-2/SSD selective state-space block is implemented anywhere in the modeling file. |
| NVIDIA-Nemotron-3.5-Lightning-30B-A3B | in its code | core | NemotronHMamba2Mixer implements the Mamba-2 chunked SSD scan with per-head scalar decay (A_log/D), used for layer type 'mamba' (linear_attention) per layers_block_type. modeling_nemotron_h.py#L366 |
| Qwen3.5-397B-A17B | not in its code | not stated | No Mamba-2/SSD selective state-space block or mamba_num_heads/ssm_state_size config is present; the recurrent layers are Gated DeltaNet, not Mamba-2. |
| Qwen3.6-35B-A3B | not in its code | not stated | The linear-attention layers implement Gated DeltaNet, not Mamba-2 SSD blocks; no mamba_num_heads/ssm_state_size mechanism is present. |
| Qwen3.8-Flash-Next | not in its code | not stated | No Mamba-2 SSD block or mamba_num_heads/ssm_state_size config exists; the recurrent mixer here is Gated DeltaNet, not Mamba-2. |
| Step-3.7-Flash | not in its code | not stated | No Mamba-2/SSD state-space block implementation exists in the given files; only Step3p7Attention (softmax) and MLP/MoE blocks are defined. |
| gpt-oss-120b | not in its code | not stated | No Mamba-2 SSM blocks are present; the model is a pure transformer with MoE FFNs. |
Evidence
6 spans quoted from the sources, strongest treatment first.
a hybrid Mamba-2 stack with sparse global Attention anchors
coremodel architecturein Nemotron 3 UltraNVIDIA
utilizing interleaved Mamba-2 and MoE layers
coremodel architecturein Nemotron 3 UltraNVIDIA
NVIDIA uses Mamba-2 for much of the sequence processing
coreunclearin Nemotron 3.5 LightningNVIDIA
interleaved Mamba-2 and MoE layers
coremodel architecturein Nemotron 3 UltraNVIDIA
a hybrid Mamba-2 stack with sparse global Attention anchors
coremodel architecturein Nemotron 3 UltraNVIDIA
Mamba-2
usedmodel architecturein Nemotron 3 UltraNVIDIA
Filed alongside
Other methods under model architecture :: token mixer :: linear attention & state space :: Mamba.