Model techniques map
ModelsGLM-5.3

Z.ai (Zhipu AI) · usage rank #9 · released 2026-08-16

GLM-5.3

What GLM-5.3’s own documents say it is built from — every method with the sentence that describes it.

OpenRouterHugging Face

Curator’s note753B MoE on the *same base model* as GLM-5.2 — every gain comes from post-training, which makes this entry the cleanest post-training-only delta in the map: +50% on Z.ai's in-house code bench, open-source SOTA on Terminal Bench 3.0, and an emergent cyber/exploitation capability the blog says scaled faster than expected. Architecture id glm_moe_dsa, 1M context, custom glm-5.3 license (not MIT, unlike 5.2). Announced 2026-08-14 for coding-plan users; weights 2026-08-16. arXiv 2602.15763 is the GLM-5 report.

Rank by open-weight tokens on OpenRouter, week of 2026-09-21.

Against the consensus recipe

Its documents state 3 of the 15 methods the field agrees on.

The recipe from the overview: the methods the most labs adopt, per stage. A missing one is something the documents do not say, not something the model lacks.

  1. Data curation

  2. Training objective

  3. Model architecture

  4. Optimization

  5. Post-training

  6. Inference & serving

Checked in its code

Its code shows 8 of the 19 architecture features checked — 8 of them are not stated in its documents.

Documents say what a lab chose to describe; the modeling code says what the checkpoint runs. Each feature below was put to a code verifier, and only the config values and source lines it cited that were found in the files are shown, linked to the line.

Read from Hugging Face Transformers (modeling_glm_moe_dsa.py, configuration_glm_moe_dsa.py) and config.json at revision aca966e4e0, by anthropic/claude-sonnet-5 on 2026-09-25.

FeatureCodeIts documentsEvidence
Grouped-query attention not in its codenot stated

num_attention_heads equals num_key_value_heads (64/64), so there is no query-head grouping over fewer KV heads; this model uses MLA instead of GQA.

num_attention_heads = 64num_key_value_heads = 64
self.num_key_value_groups = config.num_attention_heads // config.num_key_value_heads
modeling_glm_moe_dsa.py · L332
Multi-head latent attention code only in its codenot stated

Keys/values are compressed into a shared low-rank kv_lora_rank latent (with a decoupled RoPE key) and up-projected per head via kv_b_proj, DeepSeek-V2/V3 style MLA.

kv_lora_rank = 512q_lora_rank = 2048qk_rope_head_dim = 64
self.kv_a_proj_with_mqa = nn.Linear(
            self.hidden_size,
            config.kv_lora_rank + config.qk_rope_head_dim,
            bias=config.attention_bias,
        )
modeling_glm_moe_dsa.py · L339–343
kv_nope = self.kv_b_proj(kv_nope).view(key_shape).transpose(1, 2)
        k_nope, value_states = torch.split(kv_nope, [self.qk_nope_head_dim, self.v_head_dim], dim=-1)
modeling_glm_moe_dsa.py · L371–372
Sliding-window attention not in its codenot stated

No sliding_window config key or windowed mask logic appears anywhere in the modeling or configuration code; attention is full causal (with the DSA sparse mask) rather than local-window.

Interleaved sliding-window and global attention not in its codenot stated

There is no layer_types pattern distinguishing sliding vs global attention; layer_types is uniformly set to 'indexed_attention' for every layer.

if self.layer_types is None:
            self.layer_types = ["indexed_attention"] * self.num_hidden_layers
configuration_glm_moe_dsa.py · L158–159
Indexer-selected sparse attention (DeepSeek Sparse Attention) code only in its codenot stated

A lightweight GlmMoeDsaIndexer scores past tokens per head and each query attends only to the index_topk selected tokens via an additive sparse mask, DeepSeek Sparse Attention style.

index_topk = 2048index_n_heads = 32index_head_dim = 128
topk = min(self.index_topk, index_scores.shape[-1])
        return index_scores.topk(topk, dim=-1).indices.to(torch.int32)  # [B, S, topk]
modeling_glm_moe_dsa.py · L252–253
index_mask = (
                topk_indices.new_ones((batch_size, seq_length, key_states.shape[2]), dtype=torch.bool)
                .scatter(-1, topk_indices.long(), False)
                .unsqueeze(1)
            )
modeling_glm_moe_dsa.py · L432–436
Linear-attention or state-space layers alongside full attention not in its codenot stated

All layers use the same GlmMoeDsaAttention (MLA + indexer sparse attention); there is no linear-attention/state-space mixer class or layer-type switch in this file.

Gated DeltaNet layers not in its codenot stated

No gated delta-rule recurrent mixer or linear_num_key_heads/linear_conv_kernel_dim config exists in this codebase.

Mamba-2 layers not in its codenot stated

No Mamba-2/SSD selective state-space block or ssm_state_size/mamba_num_heads config is present.

Learnable attention sink not in its codenot stated

There is no learned per-head sink logit added to the attention softmax denominator anywhere in the eager or MLA attention code.

Gated attention output not in its codenot stated

The attention output is passed straight to o_proj with no sigmoid gate multiplication before the output projection.

attn_output = attn_output.reshape(batch_size, seq_length, -1).contiguous()
        attn_output = self.o_proj(attn_output)
        return attn_output, attn_weights, topk_indices
modeling_glm_moe_dsa.py · L460–462
QK normalization not in its codenot stated

Queries and keys in the main attention (q_b_proj/kv_b_proj outputs) are not normalized before the dot product; only the indexer's key gets a LayerNorm (k_norm), which is not the main QK path, and there is no use_qk_norm config.

Partial RoPE code only in its codenot stated

RoPE is applied only to the qk_rope_head_dim slice (64 of 256 total qk_head_dim) while qk_nope_head_dim carries no positional signal, the MLA decoupled-RoPE-key pattern.

qk_rope_head_dim = 64qk_nope_head_dim = 192
q_pass, q_rot = torch.split(q_states, [self.qk_nope_head_dim, self.qk_rope_head_dim], dim=-1)
modeling_glm_moe_dsa.py · L396
YaRN RoPE scaling not in its codenot stated

rope_parameters.rope_type is 'default', not 'yarn', so the yarn_apply_mscale/yarn_get_mscale code paths are not activated by this config.

rope_parameters = {"rope_theta": 8000000, "rope_type": "default"}
def yarn_apply_mscale(rope_parameters, scaling):
    if rope_parameters.get("rope_type", "default") != "default":
modeling_glm_moe_dsa.py · L299–300
Layers without positional encoding (NoPE) not in its codenot stated

Every attention layer uses the same partial-RoPE MLA formulation (qk_rope_head_dim rotated, qk_nope_head_dim not); there is no config or code path that fully skips RoPE on some softmax-attention layers while applying it on others.

Mixture of experts code only in its codenot stated

Sparse MLP layers route each token via a learned TopkRouter to 8 of 256 routed experts (n_routed_experts=256, num_experts_per_tok=8).

n_routed_experts = 256num_experts_per_tok = 8
topk_indices = torch.topk(scores_for_choice, k=self.top_k, dim=-1, sorted=False)[1]
        topk_weights = scores.gather(1, topk_indices)
modeling_glm_moe_dsa.py · L513–514
Shared expert code only in its codenot stated

Each MoE layer adds a shared_experts MLP (sized by n_shared_experts=1) that every token passes through in addition to routed experts.

n_shared_experts = 1
self.shared_experts = GlmMoeDsaMLP(
            config=config, intermediate_size=config.moe_intermediate_size * config.n_shared_experts
        )
modeling_glm_moe_dsa.py · L572–574
hidden_states = hidden_states + self.shared_experts(residuals)
modeling_glm_moe_dsa.py · L582
Auxiliary-loss-free load balancing (selection bias) code only in its codenot stated

The router adds an e_score_correction_bias buffer to sigmoid scores only for top-k/group selection (topk_method noaux_tc), while the actual weights used for combining are taken from the unbiased scores.

topk_method = "noaux_tc"
scores_for_choice = scores + self.e_score_correction_bias
modeling_glm_moe_dsa.py · L498
topk_weights = scores.gather(1, topk_indices)
modeling_glm_moe_dsa.py · L514
Multi-token prediction layers code only in its codenot stated

num_nextn_predict_layers is 1 and the quantization config's modules_to_not_convert list explicitly references an extra layer 78 (enorm/hnorm/eh_proj/shared_head) beyond the 78 main layers (0-77), indicating a declared MTP module.

num_nextn_predict_layers = 1num_hidden_layers = 78
SwiGLU feed-forward code only in its codenot stated

Both the dense MLP and the routed experts compute silu(gate_proj(x)) * up_proj(x), a SwiGLU feed-forward, matching hidden_act='silu'.

hidden_act = "silu"
down_proj = self.down_proj(self.act_fn(self.gate_proj(x)) * self.up_proj(x))
modeling_glm_moe_dsa.py · L477
gate, up = nn.functional.linear(current_state, self.gate_up_proj[expert_idx]).chunk(2, dim=-1)
            current_hidden_states = self.act_fn(gate) * up
modeling_glm_moe_dsa.py · L553–554

optimization 1

training parallelism 1

Expert Parallelism used

data curation 4

synthetic data 3

End-to-end synthetic environment generation used
Verifier synthesis without reference solutions used
Vulnerability-focused training data and environments used

data mixture & curriculum 1

Vulnerability-discovery data inclusion used

post-training 3

filed at the root 1

Extended post-training used

reinforcement learning algorithm 1

SAO with Compaction used

agentic post-training 1

Environment scaling for realistic training tasks used

inference & serving 9

decoding strategy 1

Multi-Token Prediction used

reasoning control 2

Configurable reasoning effort default
clear_thinking chat-template parameter default

KV cache management 1

Disable prefix caching for benchmarking used

inference quantization 3

FP8 default
FP8 KV-cache quantization used
NVFP4 ModelOpt re-quantization optional

inference scheduling 1

Max sequences tuning used

context management 1

Context management strategy used

software implementation 1

kernel & quantization library 1

AITER used

evaluation 13

filed at the root 4

avg@k used
Two-dimensional coding-agent evaluation used
Union-of-capabilities coverage scoring used
Zero-shot base-model baseline fallback used

benchmark 2

Disabling tool search used
Domain whitelist used

evaluation harness 2

Isolated container per rollout used
mini-swe-agent harness used

judge 4

LLM-based anti-cheating judgment used
LLM-based external API usage inspection used
Official task verifier scoring used
Rule-based anti-cheat checks used

human & real-world evaluation 1

Real-world codebase testing used

other 1

filed at the root 1

Removing Pattern-Matching-Based Anti-Cheat Checks not used

unfiled 14

post-training core
post-training scaling core
scaling post-training core
scaling post-training with more task environments, environment types, and longer training core
adding --extra-index-url to restore Docker image builds used
benchmaxxing used
Claude Code harness used
inference monitoring with request classifier and chain-of-thought monitoring used
pass@1 metric used
per-model token-per-second rescaling of timeouts used
private benchmark for evaluation used
RL training with more environments, diverse tasks, and compute used
staged release used
buying benchmark-specific training data mentioned

Sources

The curated document list for this model. "not fetched" means the URL is recorded but its text was not read in the current run.

DocumentKindPublisherStatus
z.ai/blog/glm-5.3official blogZ.ainot fetched
arxiv.org/abs/2602.15763technical reportGLM-5 Teamread
huggingface.co/zai-org/GLM-5.3model cardZ.airead
github.com/zai-org/GLM-5code repoZ.airead
docs.z.ai/guides/llm/glm-5.3vendor docsZ.airead
recipes.vllm.ai/zai-org/GLM-5.3vendor docsvLLMread
interconnects.ai/p/glm-53-how-chinese-labs-keep-stridethird party analysisInterconnects (Nathan Lambert)read
the-decoder.com/zhipu-ai-releases-glm-5-3-claims-its-the-strongest-open…newsThe Decoderread
marktechpost.com/2026/08/14/z-ai-ships-glm-5-3-without-retraining-the-b…newsMarkTechPostread