Open Models Almanac
Methodsmodel architecturetoken mixersparse attention

specific method · filed under model architecture

Gated DeepSeek Sparse Attention

A gated variant of DeepSeek Sparse Attention, with IndexCache mentioned as an accompanying feature rather than part of the method name.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See sparse attention for the methods filed beside it.

sources
3
model
1
lab adopt it
1
strongest
core

Documented in

Tencent (Hunyuan)

Evidence

3 quotes from 3 sources, one card per model, each opening with its strongest.

How sources treat it
core 3

Hy4-preview

Tencent (Hunyuan)adopted3 sources

the attention module employs Gated DeepSeek Sparse Attention (Gated DSA) with IndexCache for cross-layer sparse index reuse

coremodel architecturein Hy4-previewTencent Hy Team
2 more from this model

the attention module employs Gated DeepSeek Sparse Attention (Gated DSA) with IndexCache

coremodel architecturein Hy4-previewTencent Hy Team

the attention module employs Gated DeepSeek Sparse Attention (Gated DSA)

coremodel architecturein Hy4-previewTencent Hy Team

Further reading

Picked by hand, not extracted: where to read more, not evidence for anything on this page.

↑