Open Models Almanac
Methodsmodel architecturetoken mixerlinear attention & state spacegated delta network

general family · filed under model architecture

Gated DeltaNet–sparse Mixture-of-Experts hybrid

A hybrid architecture combining Gated DeltaNet layers with sparse Mixture-of-Experts, as distinguished from the attention method alone.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See gated delta network for the methods filed beside it.

sources
2
model
1
lab adopt it
1
strongest
core

Documented in

Alibaba (Qwen)

Evidence

2 quotes from 2 sources, one card per model, each opening with its strongest.

How sources treat it
core 2

Qwen3.5-397B-A17B

Alibaba (Qwen)adopted2 sources

Gated Delta Networks combined with sparse Mixture-of-Experts deliver high-throughput inference with minimal latency and cost overhead.

coremodel architecturein Qwen3.5-122B-A10BQwen Team
1 more from this model

Gated Delta Networks combined with sparse Mixture-of-Experts deliver high-throughput inference with minimal latency and cost overhead.

coremodel architecturein Qwen3.5-35B-A3BQwen Team
↑