Open Models Almanac
ModelsDeepSeek-V3.2

DeepSeek

DeepSeek-V3.2

What DeepSeek-V3.2’s own documents say it is built from — every method with the sentence that describes it.

Parameters
671B / 37B active
Context
160K
Attention
sparse (indexer)MLA + DeepSeek Sparse Attention (lightning indexer)
Experts
256 routed + 1 shared, top-8
Input
text
Licence
MIT
Released
2025-12-01

Against the consensus recipe

Its documents state 0 of the 18 methods the field agrees on.

The recipe from the overview: the methods the most labs adopt, per stage. A missing one is something its documents do not say, not something the model lacks.
  1. Data curation

  2. Training objective

  3. Model architecture

  4. Optimization

  5. Post-training

  6. Inference & serving

Checked in its code

Its code shows 8 of the 19 architecture features checked — 7 of them are not stated in its documents.

Sources say what a lab chose to describe; the modeling code says what the checkpoint runs. Each feature below was put to a code verifier, and only the config values and source lines it cited that were found in the files are shown, linked to the line.

Read from Hugging Face Transformers (modeling_deepseek_v32.py, configuration_deepseek_v32.py) and config.json at revision a7e62ac04e, by anthropic/claude-sonnet-5 on 2026-11-07.

FeatureCodeIts documentsVerdict
Grouped-query attention not in its codenot stated

num_attention_heads equals num_key_value_heads (128 each), so num_key_value_groups is 1 and there is no grouping; this model uses MLA…

show evidence

num_attention_heads equals num_key_value_heads (128 each), so num_key_value_groups is 1 and there is no grouping; this model uses MLA rather than GQA.

num_attention_heads = 128num_key_value_heads = 128
self.num_key_value_groups = config.num_attention_heads // config.num_key_value_heads
modeling_deepseek_v32.py · L355
Multi-head latent attention code only in its codenot stated

Attention compresses KV into a shared kv_lora_rank latent with a decoupled RoPE key, then up-projects per head via kv_b_proj…

show evidence

Attention compresses KV into a shared kv_lora_rank latent with a decoupled RoPE key, then up-projects per head via kv_b_proj, DeepSeek-V2/V3 style MLA.

kv_lora_rank = 512q_lora_rank = 1536qk_rope_head_dim = 64
compressed_kv = self.kv_a_proj_with_mqa(hidden_states)
        kv_pass, k_rot = torch.split(compressed_kv, [self.kv_lora_rank, self.qk_rope_head_dim], dim=-1)
        # Both latents are viewed as single-head, 4D tensors, as expected by `expand_kv`
        k_pass = self.kv_a_layernorm(kv_pass).view(batch_size, 1, seq_length, self.kv_lora_rank)
modeling_deepseek_v32.py · L418–421
Sliding-window attention not in its codenot stated

No sliding_window config key or windowed masking logic exists anywhere in this model's attention implementation.

Interleaved sliding-window and global attention not in its codenot stated

There is no layer_types or sliding_window_pattern distinguishing local vs global attention; every layer is uniformly 'indexed_attention'.

show evidence
if self.layer_types is None:
            self.layer_types = ["indexed_attention"] * self.num_hidden_layers
configuration_deepseek_v32.py · L138–139
Indexer-selected sparse attention (DeepSeek Sparse Attention) in its codeadopted

A DeepseekV32Indexer scores past tokens with dedicated projections and selects the top index_topk tokens per query, which are masked into…

show evidence

A DeepseekV32Indexer scores past tokens with dedicated projections and selects the top index_topk tokens per query, which are masked into the attention computation (DeepSeek Sparse Attention).

index_topk = 2048index_n_heads = 64index_head_dim = 128
topk = min(self.index_topk, index_scores.shape[-1])
        return index_scores.topk(topk, dim=-1).indices.to(torch.int32)  # [B, S, topk]
modeling_deepseek_v32.py · L240–241
Linear-attention or state-space layers alongside full attention not in its codenot stated

All layers use the same DeepseekV32Attention (MLA + indexer); there are no linear-attention/state-space mixer layers in this file.

Gated DeltaNet layers not in its codenot stated

No gated delta-rule recurrent mixer code or related config keys (linear_num_key_heads, linear_conv_kernel_dim) are present.

Mamba-2 layers not in its codenot stated

No Mamba-2/SSD selective state-space block implementation or related config keys (mamba_num_heads, ssm_state_size) exist in these files.

Learnable attention sink not in its codenot stated

The attention softmax implementation (eager_attention_forward) has no extra learned sink logit term added to the denominator.

show evidence
attn_weights = nn.functional.softmax(attn_weights, dim=-1, dtype=torch.float32).to(query.dtype)
modeling_deepseek_v32.py · L273
Gated attention output not in its codenot stated

The attention output is reshaped and passed straight to o_proj with no sigmoid gating multiplication applied.

show evidence
attn_output = attn_output.reshape(batch_size, seq_length, -1).contiguous()
        attn_output = self.o_proj(attn_output)
modeling_deepseek_v32.py · L475–476
QK normalization not in its codenot stated

Only the indexer's key gets a LayerNorm (k_norm); the main MLA query and key states (q_states, key_states) are not normalized before the…

show evidence

Only the indexer's key gets a LayerNorm (k_norm); the main MLA query and key states (q_states, key_states) are not normalized before the dot product.

query_states = torch.cat((q_pass, q_rot), dim=-1)

        key_states, value_states = self.expand_kv(k_pass, k_rot)
modeling_deepseek_v32.py · L430–432
Partial RoPE code only in its codenot stated

Only qk_rope_head_dim=64 of each head's dimension carries RoPE while qk_nope_head_dim=128 carries no position signal, both in the main…

show evidence

Only qk_rope_head_dim=64 of each head's dimension carries RoPE while qk_nope_head_dim=128 carries no position signal, both in the main attention and the decoupled MLA key.

qk_rope_head_dim = 64qk_nope_head_dim = 128
q_pass, q_rot = torch.split(q_states, [self.qk_nope_head_dim, self.qk_rope_head_dim], dim=-1)
modeling_deepseek_v32.py · L416
YaRN RoPE scaling code only in its codenot stated

rope_scaling.type is 'yarn' and the rotary embedding init dispatches to the YaRN ROPE_INIT_FUNCTIONS, with mscale applied to the attention…

show evidence

rope_scaling.type is 'yarn' and the rotary embedding init dispatches to the YaRN ROPE_INIT_FUNCTIONS, with mscale applied to the attention scaling.

rope_scaling.type = "yarn"rope_scaling.factor = 40
self.rope_type = self.config.rope_parameters["rope_type"]
        rope_init_fn: Callable = self.compute_default_rope_parameters
        if self.rope_type != "default":
            rope_init_fn = ROPE_INIT_FUNCTIONS[self.rope_type]
modeling_deepseek_v32.py · L75–78
Layers without positional encoding (NoPE) not in its codenot stated

layer_types is uniformly 'indexed_attention' for all layers and every layer applies the same partial RoPE pattern via…

show evidence

layer_types is uniformly 'indexed_attention' for all layers and every layer applies the same partial RoPE pattern via apply_rotary_pos_emb_interleave; no layer fully skips positional encoding.

if self.layer_types is None:
            self.layer_types = ["indexed_attention"] * self.num_hidden_layers
configuration_deepseek_v32.py · L138–139
Mixture of experts code only in its codenot stated

DeepseekV32MoE routes tokens to num_experts_per_tok of n_routed_experts experts via a learned sigmoid router with grouped top-k selection.

show evidencen_routed_experts = 256num_experts_per_tok = 8
topk_indices = torch.topk(scores_for_choice, k=self.top_k, dim=-1, sorted=False)[1]
        topk_weights = scores.gather(1, topk_indices)
modeling_deepseek_v32.py · L528–529
Shared expert code only in its codenot stated

DeepseekV32MoE adds a shared_experts MLP (sized by moe_intermediate_size * n_shared_experts=1) applied to every token in addition to…

show evidence

DeepseekV32MoE adds a shared_experts MLP (sized by moe_intermediate_size * n_shared_experts=1) applied to every token in addition to routed experts.

n_shared_experts = 1
self.shared_experts = DeepseekV32MLP(
            config=config, intermediate_size=config.moe_intermediate_size * config.n_shared_experts
        )
modeling_deepseek_v32.py · L587–589
Auxiliary-loss-free load balancing (selection bias) code only in its codenot stated

The router adds e_score_correction_bias to sigmoid scores only for top-k expert/group selection (noaux_tc method), while actual weights…

show evidence

The router adds e_score_correction_bias to sigmoid scores only for top-k expert/group selection (noaux_tc method), while actual weights come from the unbiased scores.

topk_method = "noaux_tc"
scores = router_logits.sigmoid()
        scores_for_choice = scores + self.e_score_correction_bias
modeling_deepseek_v32.py · L512–513
Multi-token prediction layers code unclearnot stated

The config declares num_nextn_predict_layers=1 but the given modeling file contains no MTP module class or usage of that key, so it is not…

show evidence

The config declares num_nextn_predict_layers=1 but the given modeling file contains no MTP module class or usage of that key, so it is not settled by these files.

num_nextn_predict_layers = 1
SwiGLU feed-forward code only in its codenot stated

Both the dense DeepseekV32MLP and the MoE DeepseekV32Experts compute silu(gate(x)) * up(x), i.e. SwiGLU, matching hidden_act='silu'.

show evidencehidden_act = "silu"
down_proj = self.down_proj(self.act_fn(self.gate_proj(x)) * self.up_proj(x))
modeling_deepseek_v32.py · L492

token mixer 8

softmax attention 2

multi-head latent attention 2

MQA mode of MLA core

“Therefore, we implement DSA based on the MQA (Shazeer, 2019) mode of MLA, where each latent vector (the key-value entry of MLA) will be…”

MHA and MQA modes of MLA used

“For DeepSeek-V3.1-Terminus, the MHA mode is used for training and prefilling, while the MQA mode is used for decoding.”

sparse attention 6

DeepSeek Sparse Attention core

“fine-grained sparse attention mechanism that reduces training and inference cost while preserving quality in long-context scenarios”

Sparse-attention continued pretraining with joint model and indexer optimization used

“Following indexer warm-up, we introduce the fine-grained token selection mechanism and optimize all model parameters to adapt the model to…”

sparse attention indexer 4

Lightning Indexer core

“The prototype of DSA primarily consists of two components: a lightning indexer and a fine-grained token selection mechanism.”

Fine-grained token selection mechanism core

“our fine-grained token selection mechanism retrieves only the key-value entries {c_s} corresponding to the top-k index scores.”

Dense attention warm-up with a frozen lightning indexer used

“In this stage, we keep dense attention and freeze all model parameters except for the lightning indexer.”

Detached indexer-input optimization used

“we detach the indexer input from the computational graph for separate optimization.”

channel mixer 1

mixture of experts 1

expert routing 1

Keep Routing core

“we preserve the expert routing paths used during sampling in the inference framework and enforce the same routing paths during training”

language modelling objective 1

Stricter token constraints during training used

“we imposed stricter token constraints during the training of the official DeepSeek-V3.2, aiming to optimize the trade-off between…”

auxiliary loss 2

KL alignment loss used

“Based on p_t,:, we set a KL-divergence loss as the training objective of the indexer”

Domain-specific KL regularization strength used

“In practice, we find that different domains benefit from varying strengths of KL regularization.”

data filtering 2

LLM-based refusal and inability filtering used

“we employed the DeepSeek-V32-Exp model to identify and remove samples in which the model explicitly indicated an inability or refusal to…”

Pass@100 filtering of synthesized RL tasks used

“We then perform RL on this dataset using DeepSeek-V3.2 and retain only instances with non-zero pass@100”

synthetic data 9

Large-scale agentic task synthesis pipeline core

“we developed a novel synthesis pipeline that systematically generates training data at scale. This facilitates scalable agentic…”

Automatic synthesis of task-oriented RL environments core

“To scale up agent environments and tasks in RL, we employ an automatic environment-synthesis agent that synthesizes 1,827 task-oriented…”

Automated environment, toolset, task, and solution synthesis pipeline used

“The synthesis workflow primarily consists of environment and toolset construction, task synthesis, and solution generation.”

Code-agent environment construction from GitHub issue-PR pairs used

“We constructed large-scale, executable environments for software issue resolution by mining millions of issue-Pull Request (PR) pairs from…”

Heterogeneous answer-generation agents producing diverse candidate responses used

“Multiple answer-generation agents with heterogeneous configurations (different checkpoints, system prompts, etc.) produce diverse…”

Iterative task-difficulty escalation with solution and verifier updates used

“The agent then iteratively increases the difficulty of the task and updates the corresponding solution and verification functions.”

Multi-agent pipeline for search-agent training data generation used

“We employ a multi-agent pipeline based on DeepSeek-V3.2 to generate diverse, high-quality training data.”

Question-construction agent that explores entities with search tools used

“A question-construction agent then explores each entity using search tools with configurable depth and breadth parameters, consolidating…”

Search-capable verification agent for answer validation used

“A verification agent with search capabilities validates all answers through multiple passes, retaining only samples where the ground-truth…”

filed at the root 1

Scalable reinforcement learning post-training core

“By implementing a robust RL protocol and scaling post-training compute, DeepSeek-V3.2 performs comparably to GPT-5.”

supervised fine-tuning 2

Prompt-based cold start for reasoning with tool use used

“We posit that the model possesses sufficient ability to accurately follow explicit instructions, thereby enabling the seamless…”

Reasoning-data system prompt used

“The system prompt requires the model to output the reasoning process in the tag thinking response.”

reinforcement learning algorithm 7

Group Relative Policy Optimization core

“For DeepSeek-V3.2, we still adopt Group Relative Policy Optimization (GRPO) ... as the RL training algorithm.”

Mixed-domain reinforcement learning core

“As DeepSeek-V3.2-Exp, we merge reasoning, agent, and human alignment training into one RL stage.”

Keep Routing core

“we preserve the expert routing paths used during sampling in the inference framework and enforce the same routing paths during training”

Reinforcement learning used

“These performance gains can be attributed to the increased computational resources allocated to RL training.”

Importance-sampling-corrected unbiased KL estimate used

“we correct the K3 estimator (Schulman, 2020) to obtain an unbiased KL estimate using the importance-sampling ratio between the current…”

Keep Sampling Mask used

“we preserve the truncation masks during sampling from and apply them to during training, ensuring both policies share identical action…”

Off-Policy Sequence Masking used

“we mask negative sequences that introduce significant policy divergence, as measured by the KL divergence between the data-sampling policy…”

reward modelling 4

Generative Reward Model used

“We then develop detailed evaluation rubrics across multiple quality dimensions and employ a generative reward model to score responses…”

Length penalty used

“For reasoning and agent tasks, we employ rule-based outcome reward, length penalty, and language consistency reward.”

Language-consistency reward used

“For reasoning and agent tasks, we employ rule-based outcome reward, length penalty, and language consistency reward.”

Rule-based outcome reward used

“For reasoning and agent tasks, we employ rule-based outcome reward, length penalty, and language consistency reward.”

policy distillation 1

Specialist Distillation used

“Once the specialist models are prepared, they are used to produce the domain-specific data for the final checkpoint.”

agentic post-training 3

Generate-verify-refine loop used

“In the IMO and CMO tasks, we employ a generate-verify-refine loop.”

Large-scale reinforcement learning on synthetic agentic data used

“large-scale RL on synthetic data yields substantial improvements over DeepSeek-V3.2-SFT on Tau2Bench, MCP-Mark, and MCP-Universe benchmarks.”

Training agentic policies with real-world tools used

“For tasks such as search, code engineering, and code interpretation, we employ real-world tools, including actual web search APIs, coding…”

mid-training & continual pretraining 1

Continued pretraining used

“Starting from a base checkpoint of DeepSeek-V3.1-Terminus, whose context length has been extended to 128K, we perform continued…”

2 more mentioned or evaluated
RL on synthetic agentic tasks in non-thinking mode evaluated

agentic post-training “we conduct RL only on synthetic agentic tasks in non-thinking mode.”

RL restricted to search and code environments evaluated

agentic post-training “DeepSeek-V3.2-Exp is trained with RL only in search and code environments.”

decoding strategy 3

Multi-stage candidate filtering used

“Specifically, we first sampled 500 candidate solutions for each problem, then applied a multi-stage filtering pipeline.”

Longest-thinking-trace selection used

“we selected the 50 samples with the longest thinking traces for final submission.”

Top-p and top-k sampling used

“Employing these strategies in RL training is also advantageous, as it avoids sampling extremely low-probability tokens that would be used…”

reasoning control 2

Generate-verify-refine used

“In the IMO and CMO tasks, we employ a generate-verify-refine loop.”

Reasoning-enabled boolean toggle optional

“Users can control the reasoning behaviour with the reasoning enabled boolean.”

context management 3

Context management method used

“To address this, we employ a context management method to derive the final score.”

Tool-use reasoning-context persistence and selective deletion used

“Historical reasoning content is discarded only when a new user message is introduced to the conversation.”

Test-time context management for extending token budgets used

“we introduce context management employing simple strategies to extend token budgets at test time”

agentic scaffolding 4

Thinking with tools used

“DeepSeek-V3.2 introduces significant updates to its chat template compared to prior versions. The primary changes involve a revised format…”

Cold-start prompting for reasoning in tool use used

“a straightforward strategy for integrating these two capabilities is through carefully designed prompting.”

Jupyter Notebook code interpreter used

“We utilize Jupyter Notebook as a code interpreter to address complex reasoning tasks.”

Prompt-enforced tool-call format used

“Important: ALWAYS adhere to this exact format for tool use: {TOOLCALL-FORMAT}”

7 more mentioned or evaluated
Thinking mode evaluated

reasoning control “The performance of non-thinking mode is slightly worse than the thinking mode, but still competitive.”

Discard-all context management evaluated

context management “Discard-all, which resets the context by discarding all previous tool call history”

Discard-75% evaluated

context management “Discard-75%, which discards the first 75% tool call history in the trajectory to free up spaces”

Parallel-fewest-step sampling evaluated

reasoning control “Parallel-fewest-step, which samples N independent trajectories and selects the trajectory with the fewest steps.”

Parallel test-time compute scaling evaluated

reasoning control “test-time compute can be scaled either serially through context management or in parallel”

Trajectory summarization and rollout re-initiation evaluated

context management “Summary, which summarizes the overflowed trajectory and re-initiates the rollout”

Serial test-time compute scaling through context management evaluated

reasoning control “test-time compute can be scaled either serially through context management or in parallel”

filed at the root 2

Sample-test and length-constraint filtering used

“we eliminated invalid submissions that failed to pass the provided sample test cases or exceeded the length constraints”

Standard function-calling format used

“Tool-use benchmarks are evaluated using the standard function call format”

benchmark 1

F2P/P2F acceptance criterion for executable environments used

“An environment is deemed successfully built only when applying the gold patch results in a non-zero count of false-to-positive (F2P) test…”

unfiled 2

cross-provider failover load balancing used

“we can recover by routing to another healthy provider”

Selection of submissions with the longest thinking traces used

“From the remaining valid candidates, we selected the 50 samples with the longest thinking traces for final submission.”

Further reading

Picked by hand, not extracted: where to read more, not evidence for anything on this page.

Sources

The curated source list for this model. "not fetched" means the URL is recorded but its text was not read in the current run.

SourceKindPublisherStatus
arxiv.org/abs/2512.02556technical reportDeepSeekread
huggingface.co/deepseek-ai/DeepSeek-V3.2model cardDeepSeekread
huggingface.co/deepseek-ai/DeepSeek-V3.2/blob/main/assets/paper.pdftechnical reportDeepSeekread
openrouter.ai/deepseek/deepseek-v3.2vendor docsOpenRouterread
lmsys.org/blog/2025-05-05-large-scale-epvendor docsLMSYSnot fetched
↑