Model techniques map
ModelsDeepSeek-V3.2

DeepSeek · released 2025-12-01

DeepSeek-V3.2

What DeepSeek-V3.2’s own documents say it is built from — every method with the sentence that describes it.

OpenRouterHugging Face

Curator’s noteThe V3 line's last release and the paper that introduced DeepSeek Sparse Attention — the mechanism the 2026 cohort went on to adopt wholesale (Hy4's Gated DSA, GLM's glm_moe_dsa, MiniMax's MSA lineage), which is why a 2025 model is worth carrying. Also a scaled RL post-training framework and the high-compute V3.2-Speciale variant. MIT; still in the weekly top 20 at rank 19. arXiv 2512.02556 is its report, mirrored in the repo as assets/paper.pdf.

Against the consensus recipe

Its documents state 3 of the 15 methods the field agrees on.

The recipe from the overview: the methods the most labs adopt, per stage. A missing one is something the documents do not say, not something the model lacks.

  1. Data curation

  2. Training objective

  3. Model architecture

  4. Optimization

  5. Post-training

  6. Inference & serving

Checked in its code

Its code shows 9 of the 19 architecture features checked — 8 of them are not stated in its documents.

Documents say what a lab chose to describe; the modeling code says what the checkpoint runs. Each feature below was put to a code verifier, and only the config values and source lines it cited that were found in the files are shown, linked to the line.

Read from Hugging Face Transformers (modeling_deepseek_v32.py, configuration_deepseek_v32.py) and config.json at revision a7e62ac04e, by anthropic/claude-sonnet-5 on 2026-09-25.

FeatureCodeIts documentsEvidence
Grouped-query attention not in its codenot stated

num_attention_heads equals num_key_value_heads (128/128) and attention uses MLA rather than shared KV head groups, so there is no GQA in the softmax attention layers.

num_attention_heads = 128num_key_value_heads = 128
self.num_key_value_groups = config.num_attention_heads // config.num_key_value_heads
modeling_deepseek_v32.py · L369
Multi-head latent attention code only in its codenot stated

Keys/values are compressed into a shared kv_lora_rank latent and up-projected per head via kv_b_proj, with a decoupled RoPE key, DeepSeek-V2/V3 style MLA.

kv_lora_rank = 512q_lora_rank = 1536qk_rope_head_dim = 64
self.kv_a_proj_with_mqa = nn.Linear(
            self.hidden_size,
            config.kv_lora_rank + config.qk_rope_head_dim,
            bias=config.attention_bias,
        )
modeling_deepseek_v32.py · L376–380
kv_pass, k_rot = torch.split(compressed_kv, [self.kv_lora_rank, self.qk_rope_head_dim], dim=-1)
modeling_deepseek_v32.py · L433
Sliding-window attention not in its codenot stated

No sliding_window key or windowed masking mechanism appears anywhere in the config or attention code; every layer is 'indexed_attention' with full causal masking modulated only by the sparse indexer.

Interleaved sliding-window and global attention not in its codenot stated

There is no layer_types pattern mixing sliding and global attention; all layers are uniformly 'indexed_attention'.

if self.layer_types is None:
            self.layer_types = ["indexed_attention"] * self.num_hidden_layers
configuration_deepseek_v32.py · L138–139
Indexer-selected sparse attention (DeepSeek Sparse Attention) in its code core

A DeepseekV32Indexer scores past tokens with lightweight projections and selects the top index_topk tokens per query, which are turned into a sparse attention mask.

index_topk = 2048index_n_heads = 64index_head_dim = 128
topk = min(self.index_topk, index_scores.shape[-1])
        return index_scores.topk(topk, dim=-1).indices.to(torch.int32)  # [B, S, topk]
modeling_deepseek_v32.py · L253–254
self.wq_b = nn.Linear(self.q_lora_rank, self.n_heads * self.head_dim, bias=False)
        self.wk = nn.Linear(self.hidden_size, self.head_dim, bias=False)
modeling_deepseek_v32.py · L185–186
Linear-attention or state-space layers alongside full attention not in its codenot stated

There are no linear-attention or SSM layer types; every decoder layer uses the same DeepseekV32Attention (MLA with indexer).

Gated DeltaNet layers not in its codenot stated

No gated delta rule / recurrent state mixer is implemented; only MLA attention layers exist.

Mamba-2 layers not in its codenot stated

No Mamba-2/SSD state-space block or related config keys (mamba_num_heads, ssm_state_size) are present in this model.

Learnable attention sink not in its codenot stated

The attention softmax implementation (eager_attention_forward) has no extra learned sink logit added to the denominator.

Gated attention output not in its codenot stated

The attention output is passed straight through o_proj with no sigmoid gating multiplication applied to attn_output.

attn_output = attn_output.reshape(batch_size, seq_length, -1).contiguous()
        attn_output = self.o_proj(attn_output)
        return attn_output, attn_weights
modeling_deepseek_v32.py · L489–491
QK normalization not in its codenot stated

There is no RMSNorm/LayerNorm applied to the main query/key projections in DeepseekV32Attention (only kv_a_layernorm/q_a_layernorm normalize the latent projections, and the indexer's k_norm is a separate module, not the main QK dot product); no use_qk_norm config flag exists.

Partial RoPE code only in its codenot stated

RoPE is applied only to the qk_rope_head_dim slice (64) of each 192-dim query/key head, while qk_nope_head_dim (128) carries no positional signal — DeepSeek's decoupled RoPE key.

qk_rope_head_dim = 64qk_nope_head_dim = 128
q_pass, q_rot = torch.split(q_states, [self.qk_nope_head_dim, self.qk_rope_head_dim], dim=-1)
modeling_deepseek_v32.py · L430
YaRN RoPE scaling code only in its codenot stated

rope_scaling type is set to yarn in the config and the rotary embedding/mscale code applies YaRN-specific frequency and magnitude scaling.

rope_scaling = {"beta_fast": 32, "beta_slow": 1, "factor": 40, "mscale": 1.0, "mscale_all_di…
def yarn_apply_mscale(rope_parameters, scaling):
    if rope_parameters.get("rope_type", "default") != "default":
        mscale_all_dim = rope_parameters.get("mscale_all_dim", 0)
        scaling_factor = rope_parameters["factor"]
modeling_deepseek_v32.py · L300–303
Layers without positional encoding (NoPE) not in its codenot stated

All attention layers apply the same decoupled RoPE scheme uniformly; there is no per-layer switch disabling RoPE entirely for some softmax attention layers.

Mixture of experts code only in its codenot stated

DeepseekV32MoE routes tokens to n_routed_experts=256 experts via a top-k sigmoid router (num_experts_per_tok=8) in every layer beyond first_k_dense_replace.

n_routed_experts = 256num_experts_per_tok = 8first_k_dense_replace = 3
topk_indices = torch.topk(scores_for_choice, k=self.top_k, dim=-1, sorted=False)[1]
        topk_weights = scores.gather(1, topk_indices)
modeling_deepseek_v32.py · L542–543
Shared expert code only in its codenot stated

DeepseekV32MoE adds a shared_experts MLP sized by n_shared_experts=1 that every token passes through in addition to routed experts.

n_shared_experts = 1
self.shared_experts = DeepseekV32MLP(
            config=config, intermediate_size=config.moe_intermediate_size * config.n_shared_experts
        )
modeling_deepseek_v32.py · L601–603
Auxiliary-loss-free load balancing (selection bias) code only in its codenot stated

The router adds e_score_correction_bias to sigmoid scores only for top-k expert selection (topk_method noaux_tc), matching DeepSeek-V3's bias-based load balancing.

topk_method = "noaux_tc"
scores = router_logits.sigmoid()
        scores_for_choice = scores + self.e_score_correction_bias
modeling_deepseek_v32.py · L526–527
Multi-token prediction layers code only in its codenot stated

The config declares num_nextn_predict_layers: 1, and the model's weight-loading ignores those extra MTP layer weights, indicating the checkpoint carries them.

num_nextn_predict_layers = 1
_keys_to_ignore_on_load_unexpected = [r"model\.layers\.61.*"]
modeling_deepseek_v32.py · L676
SwiGLU feed-forward code only in its codenot stated

Both the dense MLP and MoE experts compute silu(gate_proj(x)) * up_proj(x), a SwiGLU feed-forward, matching hidden_act 'silu'.

hidden_act = "silu"
down_proj = self.down_proj(self.act_fn(self.gate_proj(x)) * self.up_proj(x))
modeling_deepseek_v32.py · L506

model architecture 9

token mixer 8

softmax attention 2

channel mixer 1

mixture of experts 1

expert routing 1

Keep Routing core

training objective 3

language modelling objective 1

Stricter token constraints during training used

auxiliary loss 2

KL alignment loss used
Domain-specific KL regularization strength used

data curation 11

data filtering 2

Model-based refusal and inability filtering used
Pass@100 filtering used

synthetic data 9

Agentic data synthesis pipeline core
Automatic synthesis of task-oriented RL environments core
Coding environments from GitHub issue–PR pairs used
Environment, toolset, task, and solution synthesis pipeline used
Heterogeneous answer-generation agents used
Iterative task-difficulty escalation used
Multi-agent search-agent training-data generation used
Search-based question construction used
Search-capable answer verification agent used

post-training 20

filed at the root 1

Scalable reinforcement learning post-training core

supervised fine-tuning 2

Prompt-based cold start for tool use used
Reasoning-data system prompt used

reinforcement learning algorithm 7

Group Relative Policy Optimization core
Mixed Reinforcement Learning core
Keep Routing core
Keep Sampling Mask used
Off-Policy Sequence Masking used
Unbiased KL Estimate used
Reinforcement Learning Training used

reward modelling 4

Generative Reward Model used
Length-adjusted RL reward used
Language consistency reward used
Rule-based outcome reward used

policy distillation 1

Specialist Distillation used

agentic post-training 4

Generate-verify-refine loop used
Reinforcement learning on synthetic agentic data used
Training agentic policies with real-world tools used
Reinforcement learning restricted to search and code environments evaluated

mid-training & continual pretraining 1

Continual pretraining used

inference & serving 20

decoding strategy 3

Multi-stage candidate filtering used
Longest-trace selection used
Top-p and top-k sampling used

reasoning control 6

Generate-verify-refine loop used
reasoning-enabled inference toggle optional
Default thinking mode evaluated
Parallel-fewest-step sampling evaluated
Parallel test-time compute scaling evaluated
Serial test-time compute scaling through context management evaluated

context management 6

Thinking context management for tool use used
Context management method used
Test-time context management for extending token budgets used
Discard-all context management evaluated
Discard-75% evaluated
Trajectory summarization and rollout re-initiation evaluated

agentic scaffolding 5

Jupyter Notebook code interpreter used
Prompt-based cold start for reasoning in tool use used
Prompt-enforced tool-call format used
Thinking with tools used
Tool calls within the thinking process used

evaluation 3

filed at the root 2

Sample test and length-constraint filtering used
Standard function-calling format used

benchmark 1

F2P/P2F acceptance criterion used

unfiled 2

cross-provider failover load balancing used
Selection of submissions with the longest thinking traces used

Further reading

Picked by hand, not extracted: where to read more, not evidence for anything on this page.

Sources

The curated document list for this model. "not fetched" means the URL is recorded but its text was not read in the current run.

DocumentKindPublisherStatus
arxiv.org/abs/2512.02556technical reportDeepSeekread
huggingface.co/deepseek-ai/DeepSeek-V3.2model cardDeepSeekread
huggingface.co/deepseek-ai/DeepSeek-V3.2/blob/main/assets/paper.pdftechnical reportDeepSeekread
openrouter.ai/deepseek/deepseek-v3.2vendor docsOpenRouterread