Open Models Almanac
Methodsmodel architecturetoken mixerhybrid layer stacking

specific method · filed under model architecture

Hybrid Sparse and Linear Attention

Combines sparse attention with linear attention rather than alternating local-window and global attention layers.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See hybrid layer stacking for the methods filed beside it.

Also called Hybrid linear-sparse attention, Hybrid attention.

sources
3
model
1
lab adopt it
1
strongest
core

Documented in

Z.ai (Zhipu AI)

Evidence

3 quotes from 3 sources, one card per model, each opening with its strongest.

How sources treat it
core 3

GLM-5.3-Flash

Z.ai (Zhipu AI)adopted3 sources

Its hybrid sparse and linear attention architecture maintains accurate long-context behavior while reducing compute overhead.

coremodel architecturein GLM-5.3-FlashZ.ai
2 more from this model

For the first time in the GLM series, Z.ai combines linear and sparse attention.

coremodel architecturein GLM-5.3-FlashZ.ai

we introduce a hybrid architecture combining sparse and linear attention

coremodel architecturein GLM-5.3-FlashZ.ai
↑