Open Models Almanac
ModelsGLM-5.2

Z.ai (Zhipu AI)

GLM-5.2

What GLM-5.2’s own documents say it is built from — every method with the sentence that describes it.

Parameters
~753B / ~40B active
Context
1M
Attention
sparse (indexer)MLA + DeepSeek Sparse Attention with IndexShare
Experts
256 routed + 1 shared, top-8
Input
text
Licence
MIT
Released
2026-06-16

Against the consensus recipe

Its documents state 7 of the 18 methods the field agrees on.

The recipe from the overview: the methods the most labs adopt, per stage. A missing one is something its documents do not say, not something the model lacks.
  1. Data curation

  2. Training objective

  3. Model architecture

  4. Optimization

  5. Post-training

  6. Inference & serving

Checked in its code

Its code shows 7 of the 19 architecture features checked — 4 of them are not stated in its documents.

Sources say what a lab chose to describe; the modeling code says what the checkpoint runs. Each feature below was put to a code verifier, and only the config values and source lines it cited that were found in the files are shown, linked to the line.

Read from Hugging Face Transformers (modeling_glm_moe_dsa.py, configuration_glm_moe_dsa.py) and config.json at revision cf457fa734, by anthropic/claude-sonnet-5 on 2026-11-07.

FeatureCodeIts documentsVerdict
Grouped-query attention not in its codenot stated

num_attention_heads equals num_key_value_heads (64/64) and attention is MLA-based rather than GQA, so there is no head-sharing group.

show evidencenum_attention_heads = 64num_key_value_heads = 64
self.num_key_value_groups = config.num_attention_heads // config.num_key_value_heads
modeling_glm_moe_dsa.py · L320
Multi-head latent attention in its codeadopted

Attention down-projects hidden states into a shared kv_lora_rank latent (with a decoupled RoPE key) and up-projects per head via…

show evidence

Attention down-projects hidden states into a shared kv_lora_rank latent (with a decoupled RoPE key) and up-projects per head via kv_b_proj, DeepSeek-V2/V3 style MLA.

kv_lora_rank = 512q_lora_rank = 2048qk_rope_head_dim = 64
self.kv_a_proj_with_mqa = nn.Linear(
            self.hidden_size,
            config.kv_lora_rank + config.qk_rope_head_dim,
            bias=config.attention_bias,
        )
modeling_glm_moe_dsa.py · L327–331
kv_nope = self.kv_b_proj(kv_nope).view(key_shape).transpose(1, 2)
        k_nope, value_states = torch.split(kv_nope, [self.qk_nope_head_dim, self.v_head_dim], dim=-1)
modeling_glm_moe_dsa.py · L359–360
Sliding-window attention not in its codediscussed

No sliding_window key or windowed masking mechanism appears anywhere in the config or code; all layers use the full causal (plus indexer…

show the whole verdict

No sliding_window key or windowed masking mechanism appears anywhere in the config or code; all layers use the full causal (plus indexer top-k sparse) mask.

Interleaved sliding-window and global attention not in its codenot stated

There is no layer_types/sliding_window_pattern distinguishing local vs global attention; all 78 layers share the same "indexed_attention"…

show evidence

There is no layer_types/sliding_window_pattern distinguishing local vs global attention; all 78 layers share the same "indexed_attention" type.

if self.layer_types is None:
            self.layer_types = ["indexed_attention"] * self.num_hidden_layers
configuration_glm_moe_dsa.py · L158–159
Indexer-selected sparse attention (DeepSeek Sparse Attention) in its codeadopted

A lightweight GlmMoeDsaIndexer scores past tokens with its own projections and each query attends only to the top index_topk=2048 tokens…

show evidence

A lightweight GlmMoeDsaIndexer scores past tokens with its own projections and each query attends only to the top index_topk=2048 tokens it selects (DeepSeek Sparse Attention).

index_topk = 2048index_n_heads = 32index_head_dim = 128
topk = min(self.index_topk, index_scores.shape[-1])
        return index_scores.topk(topk, dim=-1).indices.to(torch.int32)  # [B, S, topk]
modeling_glm_moe_dsa.py · L241–242
Linear-attention or state-space layers alongside full attention not in its codenot stated

No linear-attention or SSM mixer layers exist in this file; every decoder layer uses GlmMoeDsaAttention (MLA with indexer sparsity).

Gated DeltaNet layers not in its codediscussed

No gated delta-rule recurrent mixer is implemented; attention layers are MLA/indexer-sparse, not Gated DeltaNet.

Mamba-2 layers not in its codenot stated

No Mamba-2 SSD state-space blocks are implemented in the provided modeling file.

Learnable attention sink not in its codenot stated

No learned per-head sink logit is added to the softmax denominator anywhere in the attention or indexer code.

Gated attention output not in its codenot stated

The attention output is passed straight to o_proj with no input-dependent sigmoid gate multiplication.

show evidence
attn_output = attn_output.reshape(batch_size, seq_length, -1).contiguous()
        attn_output = self.o_proj(attn_output)
        return attn_output, attn_weights, topk_indices
modeling_glm_moe_dsa.py · L448–450
QK normalization not in its codenot stated

No RMSNorm/LayerNorm is applied to the query or key states before the attention dot product in GlmMoeDsaAttention (only the indexer's…

show the whole verdict

No RMSNorm/LayerNorm is applied to the query or key states before the attention dot product in GlmMoeDsaAttention (only the indexer's separate k has a LayerNorm, which is not the main QK dot product).

Partial RoPE code only in its codenot stated

Only the qk_rope_head_dim=64 slice of each 256-dim query/key head is rotated (q_rot/q_pass split), the remaining qk_nope_head_dim=192…

show evidence

Only the qk_rope_head_dim=64 slice of each 256-dim query/key head is rotated (q_rot/q_pass split), the remaining qk_nope_head_dim=192 carries no position signal, as in DeepSeek-style decoupled RoPE.

qk_rope_head_dim = 64qk_nope_head_dim = 192
q_pass, q_rot = torch.split(q_states, [self.qk_nope_head_dim, self.qk_rope_head_dim], dim=-1)
modeling_glm_moe_dsa.py · L384
YaRN RoPE scaling not in its codenot stated

rope_parameters.rope_type is "default" (not yarn), so although yarn_apply_mscale code exists, it is a no-op for this config.

show evidencerope_parameters = {"rope_theta": 8000000, "rope_type": "default"}
def yarn_apply_mscale(rope_parameters, scaling):
    if rope_parameters.get("rope_type", "default") != "default":
modeling_glm_moe_dsa.py · L288–289
Layers without positional encoding (NoPE) not in its codenot stated

Every attention layer applies the same decoupled RoPE scheme (rotary applied to the rope slice); there is no subset of softmax attention…

show the whole verdict

Every attention layer applies the same decoupled RoPE scheme (rotary applied to the rope slice); there is no subset of softmax attention layers lacking positional encoding entirely.

Mixture of experts in its codeadopted

MoE layers (mlp_layer_types == "sparse") route tokens via GlmMoeDsaTopkRouter to top-8 of 256 routed experts.

show evidencen_routed_experts = 256num_experts_per_tok = 8
topk_indices = torch.topk(scores_for_choice, k=self.top_k, dim=-1, sorted=False)[1]
        topk_weights = scores.gather(1, topk_indices)
modeling_glm_moe_dsa.py · L501–502
Shared expert code only in its codenot stated

GlmMoeDsaMoE adds a shared_experts MLP (sized by n_shared_experts=1) that every token passes through in addition to the routed experts.

show evidencen_shared_experts = 1
self.shared_experts = GlmMoeDsaMLP(
            config=config, intermediate_size=config.moe_intermediate_size * config.n_shared_experts
        )
modeling_glm_moe_dsa.py · L560–562
Auxiliary-loss-free load balancing (selection bias) code only in its codenot stated

The router adds a per-expert e_score_correction_bias to scores used only for top-k selection (not for weighting), matching topk_method…

show evidence

The router adds a per-expert e_score_correction_bias to scores used only for top-k selection (not for weighting), matching topk_method "noaux_tc".

topk_method = "noaux_tc"
scores_for_choice = scores + self.e_score_correction_bias
modeling_glm_moe_dsa.py · L486
Multi-token prediction layers code unclearadopted

The config declares num_nextn_predict_layers=1 and weights up to layer 78 are ignored on load, but no MTP module class is present in the…

show evidence

The config declares num_nextn_predict_layers=1 and weights up to layer 78 are ignored on load, but no MTP module class is present in the given modeling file to confirm how it is built.

num_nextn_predict_layers = 1
_keys_to_ignore_on_load_unexpected = [r"model\.layers\.78.*"]
modeling_glm_moe_dsa.py · L637
SwiGLU feed-forward code only in its codenot stated

Both the dense MLP and the MoE experts compute silu(gate_proj(x)) * up_proj(x), i.e. SwiGLU, with hidden_act set to silu.

show evidencehidden_act = "silu"
down_proj = self.down_proj(self.act_fn(self.gate_proj(x)) * self.up_proj(x))
modeling_glm_moe_dsa.py · L465

token mixer 5

softmax attention 1

multi-head latent attention 1

Multi-head Latent Attention core

“MLAPO (Multi-head Latent Attention Pre-processing Optimization)”

sparse attention 4

DeepSeek Sparse Attention core

“we adopt DSA (DeepSeek Sparse Attention) [9], a novel architectural innovation that significantly reduces both training and inference costs.”

Sparse attention used

“every four sparse attention layers”

sparse attention indexer 2

IndexShare core

“We propose IndexShare, which reuses the same indexer across every four sparse attention layers, reducing per-token FLOPs by 2.9× at a 1M…”

Lightning Indexer used

“Lightning Indexer integrates score calculation, ReLU, and TopK operations into a single kernel”

channel mixer 2

mixture of experts 2

Mixture of experts core

“GLM-5 scales to 256 experts and reduces its layer count to 80 to minimize expert parallelism communication overhead.”

expert routing 1

DP-aware routing used

“We propose a DP-aware routing mechanism to preserve KV cache locality under Data Parallelism (DP) for large-scale MoE inference”

prediction head 2

Multi-Token Prediction used

“Multi-token prediction (MTP) [13;25] increases the performance of base models and acts as draft models for speculative decoding [20].”

MTP speculative decoding used

“We also improve GLM-5.2’s MTP layer for speculative decoding, increasing the acceptance length by up to 20%”

5 more mentioned or evaluated
Gated DeltaNet evaluated

gated delta network “Gated DeltaNet (GDN) [54]: A linear attention variant that replaces the quadratic softmax attention computation with a gated linear…”

Sliding Window Attention evaluated

sliding window attention “Sliding Window Attention (SWA) Interleave: A fixed alternating pattern of full-attention and windowed-attention layers applied uniformly…”

Search-Based Sliding-Window Attention Pattern evaluated

hybrid layer stacking “We introduce a search-based adaptation method that identifies the optimal subset of layers for SWA conversion while retaining full…”

SimpleGDN evaluated

gated delta network “SimpleGDN: A minimalist linearization strategy designed for maximal reuse of pre-trained weights, improving upon GDN for…”

Multi-query attention mentioned

grouped-query attention “In the future, we plan to migrate the inference backend to the training engine and uniformly adopt the Multi-Query Attention (MQA) mode of…”

optimizer 2

Muon used

“In our experiments with Muon optimizer, we find that MLA with a 576-dimension latent KV-cache cannot match the performance of GQA-8.”

Muon Split used

“The method, denoted as Muon Split, enables projection weights for different attention heads to update at different scales.”

learning-rate schedule 2

Batch-size warmup used

“including the Muon optimizer, cosine decay, and batch size warmup”

Cosine decay used

“including the Muon optimizer, cosine decay, and batch size warmup”

training precision 1

Mixed-precision training used

“substantially increase throughput via mixed-precision training/rollouts together with MTP and Prefill-Decode (PD) disaggregation”

quantization-aware training 1

INT4 Quantization-Aware Training used

“To provide better accuracy at low-precision, we apply INT4 QAT in the SFT stage.”

training stability 1

Off-policy sample filtering used

“we record the failure reason for each sample and exclude samples that fail due to environment collapse”

training parallelism 1

Expert parallelism used

“We utilized a hybrid approach combining Attention Data Parallelism (DP) and MoE Expert Parallelism (EP)”

training runtime 1

Heartbeat-driven fault tolerance used

“GLM-5 leverages slime’s heartbeat-driven fault-tolerance”

data sourcing 1

Web knowledge graph construction and question generation used

“Web Knowledge Graph (WKG) Construction and Question Generation”

data filtering 1

Three-stage question filtering used

“We apply a three-stage pipeline to balance difficulty and correctness”

synthetic data 1

Agent-based synthetic-data generation pipeline used

“we design an agentic data synthesis pipeline comprising three phases”

supervised fine-tuning 1

Rejection sampling used

“rejection sampling fine-tuning and mask fine-tuning”

reinforcement learning algorithm 4

Group Relative Policy Optimization used

“Our RL algorithm builds upon GRPO [40] and incorporates the IcePop technique [61] to mitigate the training-inference mismatch.”

IcePop used

“Our RL algorithm builds upon GRPO [40] and incorporates the IcePop technique [61] to mitigate the training-inference mismatch.”

Direct Double-sided Importance Sampling used

“we employ a Direct Double-sided Importance Sampling, which applies a token-level clipping mechanism ( ) to rollout log-probabilities…”

Group-wise policy optimization used

“We adopt the group-wise policy optimization algorithm for RL training”

reward modelling 2

Hybrid reward system used

“To supervise the diverse objectives above, we build a hybrid reward system that integrates three complementary types of reward signals…”

Multi-level reward formulation used

“We propose a multi-level reward formulation”

policy distillation 1

On-Policy Cross-Stage Distillation used

“By leveraging on-policy cross-stage distillation as the final refinement, GLM-5 effectively mitigates capability regression while…”

rollout & RL infrastructure 5

Asynchronous reinforcement learning used

“we have engineered a new asynchronous reinforcement learning infrastructure.”

SLIME used

“we developed slime, a novel asynchronous RL infrastructure that substantially improves training throughput and efficiency, enabling more…”

Token-in-Token-out (TITO) used

“First, a Token-in-Token-out (TITO) gateway eliminates re-tokenization mismatches by preserving exact action-level correspondence.”

Asynchronous Agent Reinforcement Learning algorithms used

“we present novel asynchronous Agent RL algorithms designed to enhance the quality of autonomous decision-making.”

Multi-Task Rollout Orchestrator used

“we introduce a server-based Multi-Task Rollout Orchestrator for multi-task RL training”

reasoning control 1

Thinking mode default

“GLM-5 supports controlling the thinking budget through the reasoning_effort parameter, which accepts two levels: max and high. max is the…”

KV cache management 2

Prefix caching used

“Prefix Cache (extending KV storage to system RAM)”

RadixCache used

“Context Management: Features like RadixCache (prefix sharing) and Prefix Cache”

inference quantization 4

FP8 used

“GLM-5 uses FP8 for rollout inference to reduce per-token latency”

Flex_AWQ_SSZ used

“Flex_AWQ_SSZ for scaling calibration”

QuaRot used

“Advanced algorithms like QuaRot for outlier suppression”

W4A8 quantization used

“we implemented a sophisticated W4A8 mixed-precision quantization strategy”

serving parallelism 2

Attention data parallelism used

“We utilized a hybrid approach combining Attention Data Parallelism (DP) and MoE Expert Parallelism (EP)”

Prefill-decode disaggregation used

“GLM-5, therefore, leverages slime’s Prefill–Decode (PD) disaggregation”

inference scheduling 1

Asynchronous inference and execution used

“Asynchronous Scheduling: Within vLLM, we implemented a mechanism to overlap the 'Device-to-Host' (D2H) sampling copies with the…”

inference kernel 1

Sparse Flash Attention used

“For the Sparse Flash Attention kernel, we specifically optimized for GLM-5's sparse patterns”

context management 3

Discard-all context management used

“We use a discard-all strategy as context management for BrowseComp, which is the same as DeepSeek-V3.2, and Kimi K2.5.”

Hierarchical context management used

“we combine keep-recent with Discard-all to form a hybrid Hierarchical Context Management strategy”

Keep-recent-k used

“we employ a simple Keep-recent-k strategy”

inference engine 6

vLLM optional

“vLLM (v0.23.0+)”

SGLang optional

“SGLang (v0.5.13.post1+)”

KTransformers optional

“KTransformers (v0.5.12+)”

Unsloth optional

“Unsloth (v0.1.47-beta+)”

vLLM-Ascend optional

“vLLM-Ascend”

xLLM optional

“xLLM”

training framework 1

Hugging Face Transformers optional

“Transformers (v0.5.12+)”

agent product 2

Claude Code used

“We evaluate Terminal-Bench 2.1 with Claude Code 2.1.167”

OpenHands used

“We run the SWE-Bench Pro suite with OpenHands”

judge 2

Agent-as-a-Judge used

“We conduct cross-validation between the Agent-as-a-Judge framework and human experts”

GPT-5.5 (medium) judge used

“We use GPT-5.5 (medium) as the judge model”

unfiled 4

Terminus 2 used

“We evaluate Terminal-Bench 2.1 with Terminus-2 framework”

FlashComm used

“FlashComm, which splits AllReduce operations to hide communication latency behind computation”

masking-based refinement used

“mask fine-tuning, allowing knowledge acquired during reinforcement learning to be injected back”

MLAPO used

“MLAPO fuses 13 small pre-processing operators into one 'super operator'”

Further reading

Picked by hand, not extracted: where to read more, not evidence for anything on this page.

Sources

The curated source list for this model. "not fetched" means the URL is recorded but its text was not read in the current run.

SourceKindPublisherStatus
z.ai/blog/glm-5.2official blogZ.ainot fetched
arxiv.org/abs/2602.15763technical reportGLM-5 Teamread
github.com/zai-org/GLM-5code repoZ.airead
huggingface.co/zai-org/GLM-5.2model cardZ.airead
docs.z.ai/guides/llm/glm-5.2vendor docsZ.airead
interconnects.ai/p/glm-52-is-the-step-change-for-openthird party analysisInterconnects (Nathan Lambert)read
tech-now.io/en/blogs/glm-5-2third party analysisTechNowread
docs.sglang.io/cookbook/autoregressive/GLM/GLM-5vendor docsSGLangnot fetched
lmsys.org/blog/2026-11-02-hicache-distributed-servingvendor docsLMSYSnot fetched
↑