DeepSeek · usage rank #1
DeepSeek-V4.1-Flash-1017
What DeepSeek-V4.1-Flash-1017’s own documents say it is built from — every method with the sentence that describes it.
- Parameters
- 552B / 16B active
- Context
- 1M
- Attention
- sparse (indexer)MLA + Compressed Sparse Attention 2 (Full/Reindex/Reuse modes, hierarchical indexer) + SWA, Causal Encoder-Decoder
- Experts
- 384 routed + 1 shared, top-6; plus 196B Engram memory params
- Input
- text, image
- Licence
- MIT
- Released
- 2026-10-17
Rank by open-weight tokens on OpenRouter, week of 2026-10-26.
Against the consensus recipe
Its documents state 1 of the 18 methods the field agrees on.
Data curation
- Benchmark decontaminationnot in its documents
- Synthetic data generationnot in its documents
- Corpus deduplicationnot in its documents
Training objective
- Multi-Token Predictionnot in its documents
- Knowledge insulationnot in its documents
- Quantization-aware distillationnot in its documents
Model architecture
- Mixture of expertscore
- Multi-Token Predictionnot in its documents
- Grouped-query attentionnot in its documents
Optimization
- Muonnot in its documents
- Expert parallelismnot in its documents
- Quantization-Aware Trainingnot in its documents
Post-training
- Supervised fine-tuningnot in its documents
- Asynchronous reinforcement learningnot in its documents
- Multi-Teacher On-Policy Distillationnot in its documents
Inference & serving
- Thinking modenot in its documents
- Multi-Token Predictionnot in its documents
- FP8not in its documents
Checked in its code
Its code shows 12 of the 19 architecture features checked — 11 of them are not stated in its documents.
Sources say what a lab chose to describe; the modeling code says what the checkpoint runs. Each feature below was put to a code verifier, and only the config values and source lines it cited that were found in the files are shown, linked to the line.
Read from vLLM (__init__.py, quant_config.py, dspark.py, mm_preprocess.py, model.py, engram.py, mhc.py, mega_mhc.py, decode.py, flashinfer_sparse.py, indexer.py, candidate_topk.py) and config.json at revision 1bae0e1088, by anthropic/claude-sonnet-5 on 2026-11-07.
| Feature | Code | Its documents | Verdict |
|---|---|---|---|
| Grouped-query attention | not in its code | not stated | This model uses DeepSeek-style multi-head latent attention (MLA) rather than GQA; keys/values are not projected with a separate smaller… show evidenceThis model uses DeepSeek-style multi-head latent attention (MLA) rather than GQA; keys/values are not projected with a separate smaller head count shared by query groups. text_config.num_attention_heads = 64text_config.num_key_value_heads = 1 |
| Multi-head latent attention code only | in its code | not stated | The text config defines kv_lora_rank-style low-rank KV compression (kv_source_layer_ids/compress_ratios), q_lora_rank and a decoupled RoPE… show evidenceThe text config defines kv_lora_rank-style low-rank KV compression (kv_source_layer_ids/compress_ratios), q_lora_rank and a decoupled RoPE head dim, and the code builds fused wq_a/wkv projections and a compressed KV cache consistent with DeepSeek MLA. text_config.q_lora_rank = 1280text_config.qk_rope_head_dim = 64text_config.compress_ratios = [0, 0, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1, 1, 1, 1, 1… |
| Sliding-window attention code only | in its code | not stated | sliding_window is set to 128 and the decoder uses a dedicated swa_cache_layer with a bounded window for attention layers, as referenced… show evidencesliding_window is set to 128 and the decoder uses a dedicated swa_cache_layer with a bounded window for attention layers, as referenced throughout the model code. text_config.sliding_window = 128 |
| Interleaved sliding-window and global attention code only | in its code | not stated | compress_ratios interleaves layers with compress_ratio 0 (SWA-only, no compressed/global KV) against layers with ratio 2 or 1 (which read… show evidencecompress_ratios interleaves layers with compress_ratio 0 (SWA-only, no compressed/global KV) against layers with ratio 2 or 1 (which read a compressed, effectively global, KV cache), giving a fixed interleaving of local and global-context attention. text_config.compress_ratios = [0, 0, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1, 1, 1, 1, 1… |
| Indexer-selected sparse attention (DeepSeek Sparse Attention) code only | in its code | not stated | An indexer (DeepseekV4Indexer / SparseAttnIndexer / SparseMQAIndexer) scores past tokens and each query attends only to the index_topk… show evidenceAn indexer (DeepseekV4Indexer / SparseAttnIndexer / SparseMQAIndexer) scores past tokens and each query attends only to the index_topk selected tokens, configured by index_n_heads/index_head_dim/index_topk. text_config.index_topk = 512text_config.index_n_heads = 32text_config.index_head_dim = 128 |
| Linear-attention or state-space layers alongside full attention | not in its code | not stated | No linear-attention, Mamba, or recurrent SSM mixer layers are present; all token-mixing layers use (sparse) softmax MLA attention. |
| Gated DeltaNet layers | not in its code | not stated | No gated delta-rule recurrent layers (Gated DeltaNet / Kimi Delta Attention) are implemented or configured; the model uses softmax sparse… show the whole verdictNo gated delta-rule recurrent layers (Gated DeltaNet / Kimi Delta Attention) are implemented or configured; the model uses softmax sparse MLA attention throughout. |
| Mamba-2 layers | not in its code | not stated | No Mamba-2 SSD state-space blocks are implemented or referenced in the config or code. |
| Learnable attention sink code only | in its code | not stated | Attention calls pass a learned attn_sink tensor into the sparse MLA kernels (sinks=self.attn_sink), implementing a learned per-head sink… show evidenceAttention calls pass a learned attn_sink tensor into the sparse MLA kernels (sinks=self.attn_sink), implementing a learned per-head sink logit absorbing probability mass in the softmax denominator. |
| Gated attention output | code unclear | not stated | The attention output projection code (DeepseekV4Attention / o_proj kernels) is not included in the given files, so whether an… show the whole verdictThe attention output projection code (DeepseekV4Attention / o_proj kernels) is not included in the given files, so whether an input-dependent sigmoid gate multiplies the attention output before the output projection cannot be determined. |
| QK normalization | code unclear | not stated | Engram's own query/key projections use an RMS-normalized dot product internally, but the files given do not show the main attention… show the whole verdictEngram's own query/key projections use an RMS-normalized dot product internally, but the files given do not show the main attention module's query/key projections (DeepseekV4Attention) to confirm or deny QK normalization in the softmax attention path. |
| Partial RoPE code only | in its code | not stated | qk_rope_head_dim (64) is smaller than head_dim (512), indicating only part of each query/key head carries RoPE (DeepSeek-style decoupled… show evidenceqk_rope_head_dim (64) is smaller than head_dim (512), indicating only part of each query/key head carries RoPE (DeepSeek-style decoupled RoPE key), consistent with the compress_rope_theta and rotary cos/sin cache usage seen in the code. text_config.qk_rope_head_dim = 64text_config.head_dim = 512 |
| YaRN RoPE scaling code only | in its code | not stated | rope_scaling in text_config sets rope_type to yarn with factor 16 and related YaRN parameters, applied via the attention module's… show evidencerope_scaling in text_config sets rope_type to yarn with factor 16 and related YaRN parameters, applied via the attention module's rotary_emb (cos_sin_cache) referenced throughout the code. text_config.rope_scaling = {"rope_type": "yarn", "factor": 16, "beta_fast": 32, "beta_slow": 1, "origina… |
| Layers without positional encoding (NoPE) | code unclear | not stated | No explicit no_rope_layers or nope_layer_interval config key is present, and the given attention implementation files do not show a… show the whole verdictNo explicit no_rope_layers or nope_layer_interval config key is present, and the given attention implementation files do not show a per-layer switch disabling RoPE for softmax attention layers, so this cannot be confirmed from the provided code/config. |
| Mixture of experts | in its code | adopted | Feed-forward layers route tokens to a subset (num_experts_per_tok=6) of n_routed_experts=384 expert MLPs chosen by a learned router/gate… show evidenceFeed-forward layers route tokens to a subset (num_experts_per_tok=6) of n_routed_experts=384 expert MLPs chosen by a learned router/gate, implemented via DeepseekV4MoE/RoutedExperts. text_config.n_routed_experts = 384text_config.num_experts_per_tok = 6 |
| Shared expert code only | in its code | not stated | n_shared_experts=1 configures a shared expert that every token passes through alongside the routed experts, and the code explicitly… show evidencen_shared_experts=1 configures a shared expert that every token passes through alongside the routed experts, and the code explicitly loads/pads shared_experts weights. text_config.n_shared_experts = 1 |
| Auxiliary-loss-free load balancing (selection bias) code only | in its code | not stated | topk_method is set to noaux_tc (DeepSeek-V3-style auxiliary-loss-free balancing) and the loader maps a gate bias to… show evidencetopk_method is set to noaux_tc (DeepSeek-V3-style auxiliary-loss-free balancing) and the loader maps a gate bias to e_score_correction_bias used only for expert selection. text_config.topk_method = "noaux_tc" |
| Multi-token prediction layers code only | in its code | not stated | num_nextn_predict_layers=3 declares MTP layers in the config, and the code loads/builds mtp.{0,1,2}.* draft weights/layers (e.g. in the… show evidencenum_nextn_predict_layers=3 declares MTP layers in the config, and the code loads/builds mtp.{0,1,2}.* draft weights/layers (e.g. in the DSpark/MTP draft model loader). text_config.num_nextn_predict_layers = 3 |
| SwiGLU feed-forward code only | in its code | not stated | hidden_act is silu and expert/FFN MLPs are built with gate_up_proj combining w1 (gate) and w3 (up) shards, matching a SiLU-gated SwiGLU MLP. show evidencetext_config.hidden_act = "silu" |
filed at the root 1
“the first built on the company's Causal Encoder-Decoder (CED) architecture”
channel mixer 2
mixture of experts 2
“V4.1 Flash is a sparse mixture-of-experts model from DeepSeek”
“It activates 8B parameters on input and 16B on output from a 552B-parameter backbone, an asymmetric split”
multimodal architecture 1
“visual and text embeddings trained jointly from the start of pre-training rather than added afterward”
agentic post-training 2
“improve end-to-end project development—from game creation to 3D modeling”
“uses reinforcement learning with self-feedback to improve end-to-end project development”
KV cache management 1
“Compressed KV caching cuts cache memory to roughly a quarter of the previous Flash generation”
inference scheduling 1
“we can recover by routing to another healthy provider”
agentic scaffolding 1
“It converts Messages, Chat Completions, and Responses API requests into the Conversation format, encodes them into DeepSeek V4 and V4.1…”
filed at the root 1
“The encoding folder contains a self-contained Python reference implementation (encoding.py) with test cases for multi-turn conversations…”
kernel & quantization library 1
“For production use, we additionally release deepseek-recipe, a set of Rust libraries with Python bindings that provides the same prompt…”
unfiled 1
1 mentioned or evaluated
“Please refer to the inference folder for instructions on weight conversion and running inference locally.”
Sources
The curated source list for this model. "not fetched" means the URL is recorded but its text was not read in the current run.
| Source | Kind | Publisher | Status |
|---|---|---|---|
| api-docs.deepseek.com/news/news261017 | official blog | DeepSeek | read |
| huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/blob/main/DeepSeek_V41_T… | technical report | DeepSeek | read |
| huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017 | model card | DeepSeek | read |
| openrouter.ai/deepseek/deepseek-v4.1-flash-1017 | vendor docs | OpenRouter | read |
| x.com/YingjunWu/article/2117387909183816165 | third party analysis | Yingjun Wu (Recurrence Research) | not fetched |