Model techniques map
ModelsGLM-5.2

Z.ai (Zhipu AI) · usage rank #11 · released 2026-06-16

GLM-5.2

What GLM-5.2’s own documents say it is built from — every method with the sentence that describes it.

OpenRouterHugging Face

Curator’s note~744-753B MoE, 1M context, MIT. No dedicated GLM-5.2 arXiv report; arXiv 2602.15763 is the GLM-5 report, and 5.2's own architecture delta (IndexShare over DeepSeek Sparse Attention) is described in the z.ai blog post.

Rank by open-weight tokens on OpenRouter, week of 2026-09-21.

Against the consensus recipe

Its documents state 9 of the 15 methods the field agrees on.

The recipe from the overview: the methods the most labs adopt, per stage. A missing one is something the documents do not say, not something the model lacks.

  1. Data curation

  2. Training objective

  3. Model architecture

  4. Optimization

  5. Post-training

  6. Inference & serving

Checked in its code

Its code shows 7 of the 19 architecture features checked — 4 of them are not stated in its documents.

Documents say what a lab chose to describe; the modeling code says what the checkpoint runs. Each feature below was put to a code verifier, and only the config values and source lines it cited that were found in the files are shown, linked to the line.

Read from Hugging Face Transformers (modeling_glm_moe_dsa.py, configuration_glm_moe_dsa.py) and config.json at revision cf457fa734, by anthropic/claude-sonnet-5 on 2026-09-25.

FeatureCodeIts documentsEvidence
Grouped-query attention not in its codenot stated

num_attention_heads equals num_key_value_heads (64/64) and the attention is MLA, so there is no key/value head sharing group beyond the single latent projection.

num_attention_heads = 64num_key_value_heads = 64
self.num_key_value_groups = config.num_attention_heads // config.num_key_value_heads
modeling_glm_moe_dsa.py · L332
Multi-head latent attention in its code core

Keys/values are compressed into a shared kv_lora_rank latent via kv_a_proj_with_mqa/kv_a_layernorm and up-projected per head by kv_b_proj, with a decoupled RoPE key, DeepSeek-V2/V3-style MLA.

kv_lora_rank = 512q_lora_rank = 2048qk_rope_head_dim = 64
self.kv_a_proj_with_mqa = nn.Linear(
            self.hidden_size,
            config.kv_lora_rank + config.qk_rope_head_dim,
            bias=config.attention_bias,
        )
        self.kv_a_layernorm = GlmMoeDsaRMSNorm(config.kv_lora_rank)
        self.kv_b_proj = nn.Linear(
            config.kv_lora_rank,
            self.num_heads * (self.qk_nope_head_dim + self.v_head_dim),
            bias=False,
        )
modeling_glm_moe_dsa.py · L339–349
Sliding-window attention not in its code evaluated

There is no sliding_window config key or windowed masking logic anywhere in the attention code; masking is a single causal mask ('indexed_attention') applied uniformly.

Interleaved sliding-window and global attention not in its codenot stated

All layers use the single 'indexed_attention' layer type with the same causal (plus sparse-index) mask; there is no interleaving of sliding-window and global attention layers.

self.layer_types = ["indexed_attention"] * self.num_hidden_layers
configuration_glm_moe_dsa.py · L159
Indexer-selected sparse attention (DeepSeek Sparse Attention) in its code core

GlmMoeDsaIndexer scores past tokens per head and each query attends only to the index_topk selected tokens via an additive sparse mask (DeepSeek Sparse Attention).

index_topk = 2048index_n_heads = 32index_head_dim = 128
topk = min(self.index_topk, index_scores.shape[-1])
        return index_scores.topk(topk, dim=-1).indices.to(torch.int32)  # [B, S, topk]
modeling_glm_moe_dsa.py · L252–253
Linear-attention or state-space layers alongside full attention not in its codenot stated

No linear-attention, Mamba, or state-space mixer layers exist in the code; every layer uses GlmMoeDsaAttention (MLA + indexer sparse attention).

Gated DeltaNet layers not in its code evaluated

No delta-rule recurrent state or decay gate mechanism is implemented anywhere in the modeling file.

Mamba-2 layers not in its codenot stated

No SSD/Mamba-2 selective state-space block or associated config keys (mamba_num_heads, ssm_state_size) exist in this model.

Learnable attention sink not in its codenot stated

The attention softmax in eager_attention_forward has no extra learned sink logit column added to the denominator.

attn_weights = nn.functional.softmax(attn_weights, dim=-1, dtype=torch.float32).to(query.dtype)
modeling_glm_moe_dsa.py · L285
Gated attention output not in its codenot stated

The attention output is reshaped and passed directly to o_proj with no sigmoid gate multiplication anywhere in the forward pass.

attn_output = attn_output.reshape(batch_size, seq_length, -1).contiguous()
        attn_output = self.o_proj(attn_output)
modeling_glm_moe_dsa.py · L460–461
QK normalization not in its codenot stated

Only the indexer's key projection is normalized (LayerNorm on wk output); the main attention's query/key states used for the softmax dot product (q_states/key_states) are not normalized before the attention computation.

k = self.k_norm(self.wk(hidden_states)).unsqueeze(2)  # [B, S, 1, D]
modeling_glm_moe_dsa.py · L228
Partial RoPE code only in its codenot stated

Query/key heads split into a nope part (qk_nope_head_dim=192) with no rotation and a rope part (qk_rope_head_dim=64) that gets RoPE, MLA's decoupled RoPE key style.

qk_rope_head_dim = 64qk_nope_head_dim = 192
q_pass, q_rot = torch.split(q_states, [self.qk_nope_head_dim, self.qk_rope_head_dim], dim=-1)
modeling_glm_moe_dsa.py · L396
YaRN RoPE scaling not in its codenot stated

rope_parameters.rope_type is 'default' (not 'yarn'), so although the code has yarn_apply_mscale/yarn_get_mscale helpers, this config does not enable YaRN scaling.

rope_parameters = {"rope_theta": 8000000, "rope_type": "default"}
def yarn_apply_mscale(rope_parameters, scaling):
    if rope_parameters.get("rope_type", "default") != "default":
modeling_glm_moe_dsa.py · L299–300
Layers without positional encoding (NoPE) not in its codenot stated

Every attention layer uses the same qk_nope/qk_rope split within MLA (partial RoPE per head), rather than some full-attention layers entirely lacking positional encoding while others have it; there is no no_rope_layers/nope_layer_interval config or logic.

Mixture of experts in its code core

MoE decoder layers route each token via GlmMoeDsaTopkRouter to 8 of 256 routed experts (GlmMoeDsaExperts) on layers marked 'sparse'.

n_routed_experts = 256num_experts_per_tok = 8
self.mlp = GlmMoeDsaMoE(config) if config.mlp_layer_types[layer_idx] == "sparse" else GlmMoeDsaMLP(config)
modeling_glm_moe_dsa.py · L592
Shared expert code only in its codenot stated

GlmMoeDsaMoE includes a shared_experts GlmMoeDsaMLP with intermediate size scaled by n_shared_experts=1, applied to every token in addition to routed experts.

n_shared_experts = 1
self.shared_experts = GlmMoeDsaMLP(
            config=config, intermediate_size=config.moe_intermediate_size * config.n_shared_experts
        )
modeling_glm_moe_dsa.py · L572–574
Auxiliary-loss-free load balancing (selection bias) code only in its codenot stated

GlmMoeDsaTopkRouter adds e_score_correction_bias to sigmoid scores only to select the top-k/top-group experts (topk_method 'noaux_tc'), not to weight outputs, which are gathered from the un-biased scores.

topk_method = "noaux_tc"
scores_for_choice = scores + self.e_score_correction_bias
modeling_glm_moe_dsa.py · L498
topk_weights = scores.gather(1, topk_indices)
modeling_glm_moe_dsa.py · L514
Multi-token prediction layers code unclear used

num_nextn_predict_layers=1 is set in config and there's a reference to skipping model.layers.78* on load, but no MTP module class or building logic is present in the given modeling file.

num_nextn_predict_layers = 1
_keys_to_ignore_on_load_unexpected = [r"model\.layers\.78.*"]
modeling_glm_moe_dsa.py · L648
SwiGLU feed-forward code only in its codenot stated

Both the dense MLP and routed experts compute silu(gate_proj(x)) * up_proj(x), a SwiGLU feed-forward, with hidden_act set to silu.

hidden_act = "silu"
down_proj = self.down_proj(self.act_fn(self.gate_proj(x)) * self.up_proj(x))
modeling_glm_moe_dsa.py · L477

model architecture 13

token mixer 10

softmax attention 3

sliding window attention 1

Sliding Window Attention evaluated

grouped-query attention 1

Multi-query attention mentioned

multi-head latent attention 1

Multi-Head Latent Attention core

sparse attention 4

DeepSeek Sparse Attention core
Sparse attention used

sparse attention indexer 2

IndexShare core
Lightning Indexer used

linear attention & state space 2

gated delta network 2

Gated DeltaNet evaluated
SimpleGDN evaluated

hybrid layer stacking 1

Search-Based Sliding-Window Attention Pattern evaluated

channel mixer 2

mixture of experts 2

Mixture of Experts core

expert routing 1

DP-aware routing used

prediction head 1

Multi-Token Prediction used

optimization 9

optimizer 2

Muon used
Muon Split used

learning-rate schedule 2

Cosine decay used
Batch-size warmup used

training precision 1

Mixed-precision training used

quantization-aware training 1

INT4 Quantization-Aware Training used

training stability 1

Off-policy sample filtering used

training parallelism 1

Expert Parallelism used

training runtime 1

Heartbeat-driven fault tolerance used

data curation 3

data sourcing 1

Web knowledge graph construction and question generation used

data filtering 1

Three-stage question filtering used

synthetic data 1

Agentic data synthesis pipeline used

post-training 14

supervised fine-tuning 1

Rejection sampling used

reinforcement learning algorithm 4

Group Relative Policy Optimization used
IcePop used
Direct Double-Sided Importance Sampling used
Group-Wise Policy Optimization used

reward modelling 2

Hybrid reward system used
Multi-level reward formulation used

policy distillation 1

On-Policy Cross-Stage Distillation used

rollout & RL infrastructure 6

Asynchronous reinforcement learning used
Asynchronous reinforcement learning infrastructure used
SLIME used
Token-in-token-out (TITO) used
Asynchronous Agent RL algorithms used
Multi-Task Rollout Orchestrator used

inference & serving 18

decoding strategy 2

Speculative decoding used
Multi-Token Prediction used

reasoning control 3

Configurable reasoning effort default
Default thinking mode used
Maximum thinking effort optional

KV cache management 2

Prefix caching used
RadixCache used

inference quantization 4

Flex_AWQ_SSZ used
FP8 inference used
QuaRot used
W4A8 quantization used

serving parallelism 2

Prefill-decode disaggregation used
Attention data parallelism used

inference scheduling 1

Asynchronous scheduling used

inference kernel 1

Sparse Flash Attention used

context management 3

Discard-all context management used
Hierarchical context management used
Keep-recent-k used

software implementation 9

inference engine 6

vLLM optional
SGLang optional
KTransformers optional
Unsloth optional
vLLM-Ascend optional
xLLM optional

training framework 1

Hugging Face Transformers optional

agent product 2

Claude Code used
OpenHands used

evaluation 2

judge 2

Agent-as-a-Judge used
GPT-5.5 (medium) judge model used

unfiled 4

Terminus 2 used
FlashComm used
masking-based refinement used
MLAPO used

Further reading

Picked by hand, not extracted: where to read more, not evidence for anything on this page.

Sources

The curated document list for this model. "not fetched" means the URL is recorded but its text was not read in the current run.

DocumentKindPublisherStatus
z.ai/blog/glm-5.2official blogZ.ainot fetched
arxiv.org/abs/2602.15763technical reportGLM-5 Teamread
github.com/zai-org/GLM-5code repoZ.airead
huggingface.co/zai-org/GLM-5.2model cardZ.airead
docs.z.ai/guides/llm/glm-5.2vendor docsZ.airead
interconnects.ai/p/glm-52-is-the-step-change-for-openthird party analysisInterconnects (Nathan Lambert)read
tech-now.io/en/blogs/glm-5-2third party analysisTechNowread