Z.ai (Zhipu AI) · usage rank #7
GLM-5.3
What GLM-5.3’s own documents say it is built from — every method with the sentence that describes it.
- Parameters
- ~753B / ~40B active
- Context
- 1M
- Attention
- sparse (indexer)MLA + DeepSeek Sparse Attention with IndexShare (same base as GLM-5.2)
- Experts
- 256 routed + 1 shared, top-8
- Input
- text
- Licence
- GLM-5.3 License (custom)
- Released
- 2026-08-16
Rank by open-weight tokens on OpenRouter, week of 2026-10-26.
Against the consensus recipe
Its documents state 4 of the 18 methods the field agrees on.
Data curation
- Benchmark decontaminationnot in its documents
- Synthetic data generationnot in its documents
- Corpus deduplicationnot in its documents
Training objective
- Multi-Token Predictionnot in its documents
- Knowledge insulationnot in its documents
- Quantization-aware distillationnot in its documents
Model architecture
- Mixture of expertsnot in its documents
- Multi-Token Predictionnot in its documents
- Grouped-query attentionnot in its documents
Optimization
- Muonnot in its documents
- Expert parallelismused
- Quantization-Aware Trainingnot in its documents
Post-training
- Supervised fine-tuningnot in its documents
- Asynchronous reinforcement learningnot in its documents
- Multi-Teacher On-Policy Distillationnot in its documents
Inference & serving
- Thinking modedefault
- Multi-Token Predictionused
- FP8default
Checked in its code
Its code shows 8 of the 19 architecture features checked — 8 of them are not stated in its documents.
Sources say what a lab chose to describe; the modeling code says what the checkpoint runs. Each feature below was put to a code verifier, and only the config values and source lines it cited that were found in the files are shown, linked to the line.
Read from Hugging Face Transformers (modeling_glm_moe_dsa.py, configuration_glm_moe_dsa.py) and config.json at revision ba32f8c19c, by anthropic/claude-sonnet-5 on 2026-11-07.
| Feature | Code | Its documents | Verdict |
|---|---|---|---|
| Grouped-query attention | not in its code | not stated | num_attention_heads equals num_key_value_heads (64/64) and the attention is MLA, so there is no query grouping over fewer KV heads. show evidencenum_attention_heads = 64num_key_value_heads = 64 |
| Multi-head latent attention code only | in its code | not stated | Keys/values are compressed into a shared kv_lora_rank latent via kv_a_proj_with_mqa/kv_a_layernorm and up-projected per head by kv_b_proj… show evidenceKeys/values are compressed into a shared kv_lora_rank latent via kv_a_proj_with_mqa/kv_a_layernorm and up-projected per head by kv_b_proj, with a decoupled RoPE key, DeepSeek-V2/V3 style MLA. kv_lora_rank = 512q_lora_rank = 2048qk_rope_head_dim = 64 |
| Sliding-window attention | not in its code | not stated | No sliding_window config key or windowed mask logic exists in the modeling or config files; attention uses the full causal mask with… show the whole verdictNo sliding_window config key or windowed mask logic exists in the modeling or config files; attention uses the full causal mask with indexer-based sparsity instead. |
| Interleaved sliding-window and global attention | not in its code | not stated | layer_types is uniformly "indexed_attention" for all layers with no sliding/global interleaving pattern present. show evidence |
| Indexer-selected sparse attention (DeepSeek Sparse Attention) code only | in its code | not stated | A GlmMoeDsaIndexer scores past tokens with lightweight projections and selects the top index_topk=2048 tokens per query, which is turned… show evidenceA GlmMoeDsaIndexer scores past tokens with lightweight projections and selects the top index_topk=2048 tokens per query, which is turned into the attention mask (DeepSeek Sparse Attention). index_topk = 2048index_n_heads = 32index_head_dim = 128 |
| Linear-attention or state-space layers alongside full attention | not in its code | not stated | All layers use the same GlmMoeDsaAttention (MLA with indexer sparsity); there is no linear-attention/SSM mixer variant in this file. |
| Gated DeltaNet layers | not in its code | not stated | No gated delta rule / recurrent state update code exists in the given files. |
| Mamba-2 layers | not in its code | not stated | No Mamba-2/SSD selective state-space block code exists in the given files. |
| Learnable attention sink | not in its code | not stated | No learned per-head sink logit is added to the attention softmax anywhere in the attention or eager_attention_forward code. |
| Gated attention output | not in its code | not stated | The attention output is passed straight to o_proj with no sigmoid gating of attn_output by an input-dependent gate. show evidence |
| QK normalization | not in its code | not stated | There is no RMSNorm/LayerNorm applied to the main attention q_states/key_states before the dot product (only q_a_layernorm/kv_a_layernorm… show evidenceThere is no RMSNorm/LayerNorm applied to the main attention q_states/key_states before the dot product (only q_a_layernorm/kv_a_layernorm normalize the low-rank latents, and the indexer k_norm normalizes the indexer's key, not the main attention heads). |
| Partial RoPE code only | in its code | not stated | Only qk_rope_head_dim=64 of the qk_head_dim=256 query/key dimension carries RoPE (q_rot/q_pass split), the decoupled MLA RoPE key. show evidenceqk_rope_head_dim = 64qk_nope_head_dim = 192qk_head_dim = 256 |
| YaRN RoPE scaling | not in its code | not stated | rope_parameters.rope_type is "default", not yarn, so although yarn_apply_mscale code exists it is a no-op for this config. show evidencerope_parameters = {"rope_theta": 8000000, "rope_type": "default"} |
| Layers without positional encoding (NoPE) | not in its code | not stated | Every attention layer is the same GlmMoeDsaAttention applying rotary embeddings to the rope slice; there is no per-layer toggle disabling… show the whole verdictEvery attention layer is the same GlmMoeDsaAttention applying rotary embeddings to the rope slice; there is no per-layer toggle disabling RoPE entirely for some softmax attention layers. |
| Mixture of experts code only | in its code | not stated | MoE layers (mlp_layer_types="sparse" for most of the 78 layers) route tokens via GlmMoeDsaTopkRouter to 8 of 256 routed experts. show evidencen_routed_experts = 256num_experts_per_tok = 8first_k_dense_replace = 3 |
| Shared expert code only | in its code | not stated | GlmMoeDsaMoE adds a shared_experts MLP sized by n_shared_experts=1 that every token passes through in addition to routed experts. show evidencen_shared_experts = 1 |
| Auxiliary-loss-free load balancing (selection bias) code only | in its code | not stated | The router adds e_score_correction_bias to sigmoid scores only for top-k expert/group selection (topk_method "noaux_tc"), not for… show evidenceThe router adds e_score_correction_bias to sigmoid scores only for top-k expert/group selection (topk_method "noaux_tc"), not for weighting outputs (weights are gathered from the un-biased scores). topk_method = "noaux_tc" |
| Multi-token prediction layers code only | in its code | not stated | Config declares num_nextn_predict_layers=1, and the quantization modules_to_not_convert list references layer 78 MTP-specific weights… show evidenceConfig declares num_nextn_predict_layers=1, and the quantization modules_to_not_convert list references layer 78 MTP-specific weights (eh_proj, shared_head.norm, enorm, hnorm) consistent with an extra MTP module. num_nextn_predict_layers = 1 |
| SwiGLU feed-forward code only | in its code | not stated | Both the dense MLP and expert MLPs compute silu(gate_proj(x)) * up_proj(x), a SwiGLU feed-forward, with hidden_act="silu". show evidencehidden_act = "silu" |
training parallelism 1
“--enable-expert-parallel”
synthetic data 3
“we built pipelines that synthesize environments end to end, and for a subset of tasks, the RL reward signal as well.”
“trained GLM-5.3 with data and environments built to find software vulnerabilities”
“Verifiers are synthesized without access to the reference solution, while solver trajectories are used to discover and close reward…”
data mixture & curriculum 1
“It added vulnerability-discovery data expecting better single-bug reasoning.”
filed at the root 2
“GLM-5.3 uses the same base model as GLM-5.2 — every gain comes from post-training.”
“all gains come from extended post-training alone”
reinforcement learning algorithm 1
“It carries over the RL strategies introduced in GLM-5.2, including SAO with compaction”
agentic post-training 1
“we pushed environment scaling toward tasks that look less like coding exercises and more like real units of expert work.”
decoding strategy 1
“--speculative-config.method mtp”
reasoning control 2
“GLM-5.3 supports controlling the thinking budget through the reasoning_effort parameter, which accepts three levels: low, high, and max.…”
“In the chat template for GLM-5.3, clear_thinking defaults to false if not passed. For chat scenarios, explicitly pass clear_thinking=true.”
KV cache management 1
“Add --no-enable-prefix-caching to the server command for a clean measurement.”
inference quantization 3
“the default zai-org/GLM-5.3 checkpoint is now native FP8”
“with FP8 KV cache — reaches the full 1M-token context on 8xB200.”
“only the MoE expert linears drop to NVFP4 while shared experts, attention, embeddings, and the early dense layers stay BF16.”
inference scheduling 1
“--max-num-seqs 32 — the single knob for fitting 1M context”
context management 1
“The evaluation is conducted with a maximum context length of 300,000 tokens, using a context management strategy.”
kernel & quantization library 1
“--linear-backend aiter --moe-backend aiter”
evaluation 13
filed at the root 4
“reporting avg@3 over three rollouts per task”
“The coverage result of a task is determined by taking the union of capabilities achieved across all revisions, and the average score is…”
“At different effort levels, we evaluate agents along two dimensions: end-to-end task completion rate and fine-grained checklist accuracy.”
“Runs that fail to produce a score fall back to the official zero-shot base-model baseline score.”
benchmark 2
“Tool Search is disabled, and the artifacts each agent produces are scored by the task's official separate verifier.”
“We also remove all Git-related information and apply a domain whitelist (allowing only essential domains such as pypi.org and…”
evaluation harness 2
“Each rollout runs in an isolated container built from the task's official image, and is capped at 600 agent turns with a 10-hour timeout.”
“We run DeepSWE using the mini-swe-agent harness with temperature=0.95, top_p=1.0, timeout=6h and 400K context.”
judge 4
“To prevent hacking, we use rule-based and a LLM-based judgement to prevent malicious behaviors (e.g., unauthorized pip or curl operations).”
“Instead, we use an LLM agent to inspect solutions for external API usage.”
“the artifacts each agent produces are scored by the task's official separate verifier.”
“To prevent hacking, we use rule-based and a LLM-based judgement to prevent malicious behaviors (e.g., unauthorized pip or curl operations).”
human & real-world evaluation 1
“we have been working with several security teams in China to run our models against real-world codebases.”
other 1
Not used 1
“we removed the original pattern-matching-based checks, as they produced false positives when a local vLLM endpoint was accessed through…”
unfiled 13
“Every reported gain comes from scaled post-training: more task environments, more environment types, longer training.”
“Scaling post-training is all we did for GLM-5.3.”
“Every reported gain comes from scaled post-training: more task environments, more environment types, longer training.”
“so we added --extra-index-url https://pypi.org/simple to restore successful builds.”
“Z.ai is not benchmaxxing to the point where GLM-5.3 is fried”
“We evaluate in Claude Code 2.1.207 with temperature=1.0, top_p=1, max_new_tokens=65536 with 6h timeout.”
“monitoring inference on their platforms via a request classifier and chain of thought monitoring”
“We obtain all results via the official evaluation service and report pass@1 averaged over 3 independent runs.”
“which are calculated as the API inference time rescaled by per-model tokens per second rate (per-model TPS sourced from Artificial…”
“Z.ai argues a private benchmark reduces contamination risk.”
“more environments, more diverse tasks, and more compute spent training on them”
“We are therefore taking a staged approach to release.”
1 more mentioned or evaluated
“Many companies’ data acquisition strategy is to buy data on the benchmarks they’re behind on.”
Sources
The curated source list for this model. "not fetched" means the URL is recorded but its text was not read in the current run.
| Source | Kind | Publisher | Status |
|---|---|---|---|
| z.ai/blog/glm-5.3 | official blog | Z.ai | not fetched |
| arxiv.org/abs/2602.15763 | technical report | GLM-5 Team | read |
| huggingface.co/zai-org/GLM-5.3 | model card | Z.ai | read |
| github.com/zai-org/GLM-5 | code repo | Z.ai | read |
| docs.z.ai/guides/llm/glm-5.3 | vendor docs | Z.ai | read |
| recipes.vllm.ai/zai-org/GLM-5.3 | vendor docs | vLLM | read |
| interconnects.ai/p/glm-53-how-chinese-labs-keep-stride | third party analysis | Interconnects (Nathan Lambert) | read |
| the-decoder.com/zhipu-ai-releases-glm-5-3-claims-its-the-strongest-open… | news | The Decoder | read |
| marktechpost.com/2026/08/14/z-ai-ships-glm-5-3-without-retraining-the-b… | news | MarkTechPost | read |
| lucaberton.com/blog/revolut-self-hosted-glm-b200-ai-engineer-nyc-2026 | third party analysis | Luca Berton | not fetched |