Open Models Almanac
Methodsmodel architecturetoken mixerhybrid layer stacking

general family · filed under model architecture

Hybrid Mamba-Attention

Names a hybrid Mamba-and-attention architecture without enough evidence to identify a more specific layer arrangement.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See hybrid layer stacking for the methods filed beside it.

sources
3
model
1
lab adopt it
1
strongest
core

Documented in

Evidence

4 quotes from 3 sources, one card per model, each opening with its strongest.

How sources treat it
core 4

NVIDIA-Nemotron-3-Ultra-550B-A55B

NVIDIAadopted3 sources

employing a Mixture-of-Experts hybrid Mamba-Attention architecture

coremodel architecturein Nemotron 3 UltraNVIDIA
3 more from this model

Nemotron 3 Ultra uses a MoE Hybrid Mamba-Attention architecture

coremodel architecturein Nemotron 3 UltraNVIDIA

Employs Mixture-of-Experts Hybrid Mamba-Attention architecture.

coremodel architecturein Nemotron 3 UltraNVIDIA

Mixture-of-Experts hybrid Mamba-Attention language model

coremodel architecturein Nemotron 3 UltraNVIDIA
↑