Open Models Almanac
ModelsGLM-5.3

Z.ai (Zhipu AI) · usage rank #7

GLM-5.3

What GLM-5.3’s own documents say it is built from — every method with the sentence that describes it.

Parameters
~753B / ~40B active
Context
1M
Attention
sparse (indexer)MLA + DeepSeek Sparse Attention with IndexShare (same base as GLM-5.2)
Experts
256 routed + 1 shared, top-8
Input
text
Licence
GLM-5.3 License (custom)
Released
2026-08-16

Rank by open-weight tokens on OpenRouter, week of 2026-10-26.

Against the consensus recipe

Its documents state 4 of the 18 methods the field agrees on.

The recipe from the overview: the methods the most labs adopt, per stage. A missing one is something its documents do not say, not something the model lacks.
  1. Data curation

  2. Training objective

  3. Model architecture

  4. Optimization

  5. Post-training

  6. Inference & serving

Checked in its code

Its code shows 8 of the 19 architecture features checked — 8 of them are not stated in its documents.

Sources say what a lab chose to describe; the modeling code says what the checkpoint runs. Each feature below was put to a code verifier, and only the config values and source lines it cited that were found in the files are shown, linked to the line.

Read from Hugging Face Transformers (modeling_glm_moe_dsa.py, configuration_glm_moe_dsa.py) and config.json at revision ba32f8c19c, by anthropic/claude-sonnet-5 on 2026-11-07.

FeatureCodeIts documentsVerdict
Grouped-query attention not in its codenot stated

num_attention_heads equals num_key_value_heads (64/64) and the attention is MLA, so there is no query grouping over fewer KV heads.

show evidencenum_attention_heads = 64num_key_value_heads = 64
self.num_key_value_groups = config.num_attention_heads // config.num_key_value_heads
modeling_glm_moe_dsa.py · L320
Multi-head latent attention code only in its codenot stated

Keys/values are compressed into a shared kv_lora_rank latent via kv_a_proj_with_mqa/kv_a_layernorm and up-projected per head by kv_b_proj…

show evidence

Keys/values are compressed into a shared kv_lora_rank latent via kv_a_proj_with_mqa/kv_a_layernorm and up-projected per head by kv_b_proj, with a decoupled RoPE key, DeepSeek-V2/V3 style MLA.

kv_lora_rank = 512q_lora_rank = 2048qk_rope_head_dim = 64
self.kv_a_proj_with_mqa = nn.Linear(
            self.hidden_size,
            config.kv_lora_rank + config.qk_rope_head_dim,
            bias=config.attention_bias,
        )
        self.kv_a_layernorm = GlmMoeDsaRMSNorm(config.kv_lora_rank)
modeling_glm_moe_dsa.py · L327–332
kv_nope = self.kv_b_proj(kv_nope).view(key_shape).transpose(1, 2)
        k_nope, value_states = torch.split(kv_nope, [self.qk_nope_head_dim, self.v_head_dim], dim=-1)
modeling_glm_moe_dsa.py · L359–360
Sliding-window attention not in its codenot stated

No sliding_window config key or windowed mask logic exists in the modeling or config files; attention uses the full causal mask with…

show the whole verdict

No sliding_window config key or windowed mask logic exists in the modeling or config files; attention uses the full causal mask with indexer-based sparsity instead.

Interleaved sliding-window and global attention not in its codenot stated

layer_types is uniformly "indexed_attention" for all layers with no sliding/global interleaving pattern present.

show evidence
        if self.layer_types is None:
            self.layer_types = ["indexed_attention"] * self.num_hidden_layers
configuration_glm_moe_dsa.py · L158–159
Indexer-selected sparse attention (DeepSeek Sparse Attention) code only in its codenot stated

A GlmMoeDsaIndexer scores past tokens with lightweight projections and selects the top index_topk=2048 tokens per query, which is turned…

show evidence

A GlmMoeDsaIndexer scores past tokens with lightweight projections and selects the top index_topk=2048 tokens per query, which is turned into the attention mask (DeepSeek Sparse Attention).

index_topk = 2048index_n_heads = 32index_head_dim = 128
topk = min(self.index_topk, index_scores.shape[-1])
        return index_scores.topk(topk, dim=-1).indices.to(torch.int32)  # [B, S, topk]
modeling_glm_moe_dsa.py · L241–242
Linear-attention or state-space layers alongside full attention not in its codenot stated

All layers use the same GlmMoeDsaAttention (MLA with indexer sparsity); there is no linear-attention/SSM mixer variant in this file.

Gated DeltaNet layers not in its codenot stated

No gated delta rule / recurrent state update code exists in the given files.

Mamba-2 layers not in its codenot stated

No Mamba-2/SSD selective state-space block code exists in the given files.

Learnable attention sink not in its codenot stated

No learned per-head sink logit is added to the attention softmax anywhere in the attention or eager_attention_forward code.

Gated attention output not in its codenot stated

The attention output is passed straight to o_proj with no sigmoid gating of attn_output by an input-dependent gate.

show evidence
attn_output = attn_output.reshape(batch_size, seq_length, -1).contiguous()
        attn_output = self.o_proj(attn_output)
        return attn_output, attn_weights, topk_indices
modeling_glm_moe_dsa.py · L448–450
QK normalization not in its codenot stated

There is no RMSNorm/LayerNorm applied to the main attention q_states/key_states before the dot product (only q_a_layernorm/kv_a_layernorm…

show evidence

There is no RMSNorm/LayerNorm applied to the main attention q_states/key_states before the dot product (only q_a_layernorm/kv_a_layernorm normalize the low-rank latents, and the indexer k_norm normalizes the indexer's key, not the main attention heads).

q_resid = self.q_a_layernorm(self.q_a_proj(hidden_states))
        q_states = self.q_b_proj(q_resid).view(query_shape).transpose(1, 2)
modeling_glm_moe_dsa.py · L382–383
Partial RoPE code only in its codenot stated

Only qk_rope_head_dim=64 of the qk_head_dim=256 query/key dimension carries RoPE (q_rot/q_pass split), the decoupled MLA RoPE key.

show evidenceqk_rope_head_dim = 64qk_nope_head_dim = 192qk_head_dim = 256
q_pass, q_rot = torch.split(q_states, [self.qk_nope_head_dim, self.qk_rope_head_dim], dim=-1)
modeling_glm_moe_dsa.py · L384
YaRN RoPE scaling not in its codenot stated

rope_parameters.rope_type is "default", not yarn, so although yarn_apply_mscale code exists it is a no-op for this config.

show evidencerope_parameters = {"rope_theta": 8000000, "rope_type": "default"}
def yarn_apply_mscale(rope_parameters, scaling):
    if rope_parameters.get("rope_type", "default") != "default":
modeling_glm_moe_dsa.py · L288–289
Layers without positional encoding (NoPE) not in its codenot stated

Every attention layer is the same GlmMoeDsaAttention applying rotary embeddings to the rope slice; there is no per-layer toggle disabling…

show the whole verdict

Every attention layer is the same GlmMoeDsaAttention applying rotary embeddings to the rope slice; there is no per-layer toggle disabling RoPE entirely for some softmax attention layers.

Mixture of experts code only in its codenot stated

MoE layers (mlp_layer_types="sparse" for most of the 78 layers) route tokens via GlmMoeDsaTopkRouter to 8 of 256 routed experts.

show evidencen_routed_experts = 256num_experts_per_tok = 8first_k_dense_replace = 3
self.mlp = GlmMoeDsaMoE(config) if config.mlp_layer_types[layer_idx] == "sparse" else GlmMoeDsaMLP(config)
modeling_glm_moe_dsa.py · L580
Shared expert code only in its codenot stated

GlmMoeDsaMoE adds a shared_experts MLP sized by n_shared_experts=1 that every token passes through in addition to routed experts.

show evidencen_shared_experts = 1
self.shared_experts = GlmMoeDsaMLP(
            config=config, intermediate_size=config.moe_intermediate_size * config.n_shared_experts
        )
modeling_glm_moe_dsa.py · L560–562
Auxiliary-loss-free load balancing (selection bias) code only in its codenot stated

The router adds e_score_correction_bias to sigmoid scores only for top-k expert/group selection (topk_method "noaux_tc"), not for…

show evidence

The router adds e_score_correction_bias to sigmoid scores only for top-k expert/group selection (topk_method "noaux_tc"), not for weighting outputs (weights are gathered from the un-biased scores).

topk_method = "noaux_tc"
scores_for_choice = scores + self.e_score_correction_bias
modeling_glm_moe_dsa.py · L486
topk_weights = scores.gather(1, topk_indices)
modeling_glm_moe_dsa.py · L502
Multi-token prediction layers code only in its codenot stated

Config declares num_nextn_predict_layers=1, and the quantization modules_to_not_convert list references layer 78 MTP-specific weights…

show evidence

Config declares num_nextn_predict_layers=1, and the quantization modules_to_not_convert list references layer 78 MTP-specific weights (eh_proj, shared_head.norm, enorm, hnorm) consistent with an extra MTP module.

num_nextn_predict_layers = 1
SwiGLU feed-forward code only in its codenot stated

Both the dense MLP and expert MLPs compute silu(gate_proj(x)) * up_proj(x), a SwiGLU feed-forward, with hidden_act="silu".

show evidencehidden_act = "silu"
def forward(self, x):
        down_proj = self.down_proj(self.act_fn(self.gate_proj(x)) * self.up_proj(x))
        return down_proj
modeling_glm_moe_dsa.py · L464–466

training parallelism 1

Expert parallelism used

“--enable-expert-parallel”

synthetic data 3

Synthetic environment generation pipelines used

“we built pipelines that synthesize environments end to end, and for a subset of tasks, the RL reward signal as well.”

Training with vulnerability-focused data and environments used

“trained GLM-5.3 with data and environments built to find software vulnerabilities”

Verifier synthesis without reference solution used

“Verifiers are synthesized without access to the reference solution, while solver trajectories are used to discover and close reward…”

data mixture & curriculum 1

Vulnerability-discovery training data used

“It added vulnerability-discovery data expecting better single-bug reasoning.”

filed at the root 2

post-training core

“GLM-5.3 uses the same base model as GLM-5.2 — every gain comes from post-training.”

Extended post-training used

“all gains come from extended post-training alone”

reinforcement learning algorithm 1

SAO with compaction used

“It carries over the RL strategies introduced in GLM-5.2, including SAO with compaction”

agentic post-training 1

Environment scaling for realistic training tasks used

“we pushed environment scaling toward tasks that look less like coding exercises and more like real units of expert work.”

decoding strategy 1

Multi-Token Prediction used

“--speculative-config.method mtp”

reasoning control 2

Thinking mode default

“GLM-5.3 supports controlling the thinking budget through the reasoning_effort parameter, which accepts three levels: low, high, and max.…”

clear_thinking chat-template flag default

“In the chat template for GLM-5.3, clear_thinking defaults to false if not passed. For chat scenarios, explicitly pass clear_thinking=true.”

KV cache management 1

Disabling prefix caching for benchmarking used

“Add --no-enable-prefix-caching to the server command for a clean measurement.”

inference quantization 3

FP8 default

“the default zai-org/GLM-5.3 checkpoint is now native FP8”

FP8 KV-cache quantization used

“with FP8 KV cache — reaches the full 1M-token context on 8xB200.”

NVFP4 quantization of MoE expert linears optional

“only the MoE expert linears drop to NVFP4 while shared experts, attention, embeddings, and the early dense layers stay BF16.”

inference scheduling 1

Maximum-sequence tuning used

“--max-num-seqs 32 — the single knob for fitting 1M context”

context management 1

Context management method used

“The evaluation is conducted with a maximum context length of 300,000 tokens, using a context management strategy.”

kernel & quantization library 1

AITER backend used

“--linear-backend aiter --moe-backend aiter”

filed at the root 4

avg@3 used

“reporting avg@3 over three rollouts per task”

Coverage scoring by union across revisions used

“The coverage result of a task is determined by taking the union of capabilities achieved across all revisions, and the average score is…”

Two-dimensional coding-agent evaluation used

“At different effort levels, we evaluate agents along two dimensions: end-to-end task completion rate and fine-grained checklist accuracy.”

Zero-shot baseline fallback used

“Runs that fail to produce a score fall back to the official zero-shot base-model baseline score.”

benchmark 2

Disabling tool search in evaluation used

“Tool Search is disabled, and the artifacts each agent produces are scored by the task's official separate verifier.”

Domain whitelist for agent environments used

“We also remove all Git-related information and apply a domain whitelist (allowing only essential domains such as pypi.org and…”

evaluation harness 2

Isolated-container-per-rollout execution used

“Each rollout runs in an isolated container built from the task's official image, and is capped at 600 agent turns with a 10-hour timeout.”

mini-swe-agent harness used

“We run DeepSWE using the mini-swe-agent harness with temperature=0.95, top_p=1.0, timeout=6h and 400K context.”

judge 4

LLM-as-a-Judge used

“To prevent hacking, we use rule-based and a LLM-based judgement to prevent malicious behaviors (e.g., unauthorized pip or curl operations).”

LLM-based external API usage inspection used

“Instead, we use an LLM agent to inspect solutions for external API usage.”

Official task verifier scoring used

“the artifacts each agent produces are scored by the task's official separate verifier.”

Rule-based anti-cheat checks used

“To prevent hacking, we use rule-based and a LLM-based judgement to prevent malicious behaviors (e.g., unauthorized pip or curl operations).”

human & real-world evaluation 1

Real-world codebase testing with security teams used

“we have been working with several security teams in China to run our models against real-world codebases.”

Not used 1

Removing pattern-matching-based anti-cheat checks not used

“we removed the original pattern-matching-based checks, as they produced false positives when a local vLLM endpoint was accessed through…”

unfiled 13

post-training scaling core

“Every reported gain comes from scaled post-training: more task environments, more environment types, longer training.”

scaling post-training core

“Scaling post-training is all we did for GLM-5.3.”

scaling post-training with more task environments, environment types, and longer training core

“Every reported gain comes from scaled post-training: more task environments, more environment types, longer training.”

adding --extra-index-url to restore Docker image builds used

“so we added --extra-index-url https://pypi.org/simple to restore successful builds.”

benchmaxxing used

“Z.ai is not benchmaxxing to the point where GLM-5.3 is fried”

Claude Code harness used

“We evaluate in Claude Code 2.1.207 with temperature=1.0, top_p=1, max_new_tokens=65536 with 6h timeout.”

inference monitoring with request classifier and chain-of-thought monitoring used

“monitoring inference on their platforms via a request classifier and chain of thought monitoring”

pass@1 metric used

“We obtain all results via the official evaluation service and report pass@1 averaged over 3 independent runs.”

per-model token-per-second rescaling of timeouts used

“which are calculated as the API inference time rescaled by per-model tokens per second rate (per-model TPS sourced from Artificial…”

private benchmark for evaluation used

“Z.ai argues a private benchmark reduces contamination risk.”

RL training with more environments, diverse tasks, and compute used

“more environments, more diverse tasks, and more compute spent training on them”

staged release used

“We are therefore taking a staged approach to release.”

1 more mentioned or evaluated
buying benchmark-specific training data mentioned

“Many companies’ data acquisition strategy is to buy data on the benchmarks they’re behind on.”

Sources

The curated source list for this model. "not fetched" means the URL is recorded but its text was not read in the current run.

SourceKindPublisherStatus
z.ai/blog/glm-5.3official blogZ.ainot fetched
arxiv.org/abs/2602.15763technical reportGLM-5 Teamread
huggingface.co/zai-org/GLM-5.3model cardZ.airead
github.com/zai-org/GLM-5code repoZ.airead
docs.z.ai/guides/llm/glm-5.3vendor docsZ.airead
recipes.vllm.ai/zai-org/GLM-5.3vendor docsvLLMread
interconnects.ai/p/glm-53-how-chinese-labs-keep-stridethird party analysisInterconnects (Nathan Lambert)read
the-decoder.com/zhipu-ai-releases-glm-5-3-claims-its-the-strongest-open…newsThe Decoderread
marktechpost.com/2026/08/14/z-ai-ships-glm-5-3-without-retraining-the-b…newsMarkTechPostread
lucaberton.com/blog/revolut-self-hosted-glm-b200-ai-engineer-nyc-2026third party analysisLuca Bertonnot fetched
↑