Open Models Almanac
ModelsDeepSeek-V4.1-Flash-1017

DeepSeek · usage rank #1

DeepSeek-V4.1-Flash-1017

What DeepSeek-V4.1-Flash-1017’s own documents say it is built from — every method with the sentence that describes it.

Parameters
552B / 16B active
Context
1M
Attention
sparse (indexer)MLA + Compressed Sparse Attention 2 (Full/Reindex/Reuse modes, hierarchical indexer) + SWA, Causal Encoder-Decoder
Experts
384 routed + 1 shared, top-6; plus 196B Engram memory params
Input
text, image
Licence
MIT
Released
2026-10-17

Rank by open-weight tokens on OpenRouter, week of 2026-10-26.

Against the consensus recipe

Its documents state 1 of the 18 methods the field agrees on.

The recipe from the overview: the methods the most labs adopt, per stage. A missing one is something its documents do not say, not something the model lacks.
  1. Data curation

  2. Training objective

  3. Model architecture

  4. Optimization

  5. Post-training

  6. Inference & serving

Checked in its code

Its code shows 12 of the 19 architecture features checked — 11 of them are not stated in its documents.

Sources say what a lab chose to describe; the modeling code says what the checkpoint runs. Each feature below was put to a code verifier, and only the config values and source lines it cited that were found in the files are shown, linked to the line.

Read from vLLM (__init__.py, quant_config.py, dspark.py, mm_preprocess.py, model.py, engram.py, mhc.py, mega_mhc.py, decode.py, flashinfer_sparse.py, indexer.py, candidate_topk.py) and config.json at revision 1bae0e1088, by anthropic/claude-sonnet-5 on 2026-11-07.

FeatureCodeIts documentsVerdict
Grouped-query attention not in its codenot stated

This model uses DeepSeek-style multi-head latent attention (MLA) rather than GQA; keys/values are not projected with a separate smaller…

show evidence

This model uses DeepSeek-style multi-head latent attention (MLA) rather than GQA; keys/values are not projected with a separate smaller head count shared by query groups.

text_config.num_attention_heads = 64text_config.num_key_value_heads = 1
Multi-head latent attention code only in its codenot stated

The text config defines kv_lora_rank-style low-rank KV compression (kv_source_layer_ids/compress_ratios), q_lora_rank and a decoupled RoPE…

show evidence

The text config defines kv_lora_rank-style low-rank KV compression (kv_source_layer_ids/compress_ratios), q_lora_rank and a decoupled RoPE head dim, and the code builds fused wq_a/wkv projections and a compressed KV cache consistent with DeepSeek MLA.

text_config.q_lora_rank = 1280text_config.qk_rope_head_dim = 64text_config.compress_ratios = [0, 0, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1, 1, 1, 1, 1…
("attn.fused_wqa_wkv", "attn.wq_a", 0),
            ("attn.fused_wqa_wkv", "attn.wkv", 1),
model.py · L1371–1372
Sliding-window attention code only in its codenot stated

sliding_window is set to 128 and the decoder uses a dedicated swa_cache_layer with a bounded window for attention layers, as referenced…

show evidence

sliding_window is set to 128 and the decoder uses a dedicated swa_cache_layer with a bounded window for attention layers, as referenced throughout the model code.

text_config.sliding_window = 128
self.decoder_replay_layers = DecoderReplayLayers(
                config.sliding_window,
                self._run_replay_layers,
model.py · L901–903
Interleaved sliding-window and global attention code only in its codenot stated

compress_ratios interleaves layers with compress_ratio 0 (SWA-only, no compressed/global KV) against layers with ratio 2 or 1 (which read…

show evidence

compress_ratios interleaves layers with compress_ratio 0 (SWA-only, no compressed/global KV) against layers with ratio 2 or 1 (which read a compressed, effectively global, KV cache), giving a fixed interleaving of local and global-context attention.

text_config.compress_ratios = [0, 0, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1, 1, 1, 1, 1…
swa_only = self.compress_ratio == 0
        # SWA-only layers have no compressed KV cache; consumers read the kv
        # source's cache.
        self_kv_cache = None if swa_only else self._compressed_kv_cache()
flashinfer_sparse.py · L313–316
Indexer-selected sparse attention (DeepSeek Sparse Attention) code only in its codenot stated

An indexer (DeepseekV4Indexer / SparseAttnIndexer / SparseMQAIndexer) scores past tokens and each query attends only to the index_topk…

show evidence

An indexer (DeepseekV4Indexer / SparseAttnIndexer / SparseMQAIndexer) scores past tokens and each query attends only to the index_topk selected tokens, configured by index_n_heads/index_head_dim/index_topk.

text_config.index_topk = 512text_config.index_n_heads = 32text_config.index_head_dim = 128
get_indexer_topk(op.topk_backend)(
        logits,
        seq_lens,
        1,
        topk_rows,
        op.topk_tokens,
        plan.max_seq_len,
    )
indexer.py · L378–385
Linear-attention or state-space layers alongside full attention not in its codenot stated

No linear-attention, Mamba, or recurrent SSM mixer layers are present; all token-mixing layers use (sparse) softmax MLA attention.

Gated DeltaNet layers not in its codenot stated

No gated delta-rule recurrent layers (Gated DeltaNet / Kimi Delta Attention) are implemented or configured; the model uses softmax sparse…

show the whole verdict

No gated delta-rule recurrent layers (Gated DeltaNet / Kimi Delta Attention) are implemented or configured; the model uses softmax sparse MLA attention throughout.

Mamba-2 layers not in its codenot stated

No Mamba-2 SSD state-space blocks are implemented or referenced in the config or code.

Learnable attention sink code only in its codenot stated

Attention calls pass a learned attn_sink tensor into the sparse MLA kernels (sinks=self.attn_sink), implementing a learned per-head sink…

show evidence

Attention calls pass a learned attn_sink tensor into the sparse MLA kernels (sinks=self.attn_sink), implementing a learned per-head sink logit absorbing probability mass in the softmax denominator.

out=decode_output,
                bmm1_scale=bmm1_scale,
                bmm2_scale=bmm2_scale,
                sinks=self.attn_sink,
                cum_seq_lens_q=decode_cu,
flashinfer_sparse.py · L540–544
Gated attention output code unclearnot stated

The attention output projection code (DeepseekV4Attention / o_proj kernels) is not included in the given files, so whether an…

show the whole verdict

The attention output projection code (DeepseekV4Attention / o_proj kernels) is not included in the given files, so whether an input-dependent sigmoid gate multiplies the attention output before the output projection cannot be determined.

QK normalization code unclearnot stated

Engram's own query/key projections use an RMS-normalized dot product internally, but the files given do not show the main attention…

show the whole verdict

Engram's own query/key projections use an RMS-normalized dot product internally, but the files given do not show the main attention module's query/key projections (DeepseekV4Attention) to confirm or deny QK normalization in the softmax attention path.

Partial RoPE code only in its codenot stated

qk_rope_head_dim (64) is smaller than head_dim (512), indicating only part of each query/key head carries RoPE (DeepSeek-style decoupled…

show evidence

qk_rope_head_dim (64) is smaller than head_dim (512), indicating only part of each query/key head carries RoPE (DeepSeek-style decoupled RoPE key), consistent with the compress_rope_theta and rotary cos/sin cache usage seen in the code.

text_config.qk_rope_head_dim = 64text_config.head_dim = 512
attn.rotary_emb.cos_sin_cache,
        attn.swa_cache_layer.block_size,
dspark.py · L284–285
YaRN RoPE scaling code only in its codenot stated

rope_scaling in text_config sets rope_type to yarn with factor 16 and related YaRN parameters, applied via the attention module's…

show evidence

rope_scaling in text_config sets rope_type to yarn with factor 16 and related YaRN parameters, applied via the attention module's rotary_emb (cos_sin_cache) referenced throughout the code.

text_config.rope_scaling = {"rope_type": "yarn", "factor": 16, "beta_fast": 32, "beta_slow": 1, "origina…
attn.rotary_emb.cos_sin_cache,
dspark.py · L284
Layers without positional encoding (NoPE) code unclearnot stated

No explicit no_rope_layers or nope_layer_interval config key is present, and the given attention implementation files do not show a…

show the whole verdict

No explicit no_rope_layers or nope_layer_interval config key is present, and the given attention implementation files do not show a per-layer switch disabling RoPE for softmax attention layers, so this cannot be confirmed from the provided code/config.

Mixture of experts in its codeadopted

Feed-forward layers route tokens to a subset (num_experts_per_tok=6) of n_routed_experts=384 expert MLPs chosen by a learned router/gate…

show evidence

Feed-forward layers route tokens to a subset (num_experts_per_tok=6) of n_routed_experts=384 expert MLPs chosen by a learned router/gate, implemented via DeepseekV4MoE/RoutedExperts.

text_config.n_routed_experts = 384text_config.num_experts_per_tok = 6
config = vllm_config.model_config.hf_config
        n_routed_experts = config.n_routed_experts
        n_activated_experts = config.num_experts_per_tok
model.py · L139–141
Shared expert code only in its codenot stated

n_shared_experts=1 configures a shared expert that every token passes through alongside the routed experts, and the code explicitly…

show evidence

n_shared_experts=1 configures a shared expert that every token passes through alongside the routed experts, and the code explicitly loads/pads shared_experts weights.

text_config.n_shared_experts = 1
if ".shared_experts." in name:
                if is_pp_missing_parameter(name, self):
                    continue
                experts = self.layers[extract_layer_index(name)].ffn.experts
model.py · L1395–1398
Auxiliary-loss-free load balancing (selection bias) code only in its codenot stated

topk_method is set to noaux_tc (DeepSeek-V3-style auxiliary-loss-free balancing) and the loader maps a gate bias to…

show evidence

topk_method is set to noaux_tc (DeepSeek-V3-style auxiliary-loss-free balancing) and the loader maps a gate bias to e_score_correction_bias used only for expert selection.

text_config.topk_method = "noaux_tc"
if name.endswith(".ffn.gate.bias"):
                    name = name.replace(
                        ".ffn.gate.bias", ".ffn.gate.e_score_correction_bias"
                    )
dspark.py · L519–522
Multi-token prediction layers code only in its codenot stated

num_nextn_predict_layers=3 declares MTP layers in the config, and the code loads/builds mtp.{0,1,2}.* draft weights/layers (e.g. in the…

show evidence

num_nextn_predict_layers=3 declares MTP layers in the config, and the code loads/builds mtp.{0,1,2}.* draft weights/layers (e.g. in the DSpark/MTP draft model loader).

text_config.num_nextn_predict_layers = 3
self.num_dspark_layers = (
            getattr(config, "n_mtp_layers", None)
            or getattr(config, "num_nextn_predict_layers", None)
            or 3
        )
dspark.py · L90–94
SwiGLU feed-forward code only in its codenot stated

hidden_act is silu and expert/FFN MLPs are built with gate_up_proj combining w1 (gate) and w3 (up) shards, matching a SiLU-gated SwiGLU MLP.

show evidencetext_config.hidden_act = "silu"
("gate_up_proj", "w1", 0),
            ("gate_up_proj", "w3", 1),
model.py · L1369–1370

filed at the root 1

Causal Encoder-Decoder (CED) core

“the first built on the company's Causal Encoder-Decoder (CED) architecture”

channel mixer 2

mixture of experts 2

Mixture of experts core

“V4.1 Flash is a sparse mixture-of-experts model from DeepSeek”

Asymmetric input/output parameter activation core

“It activates 8B parameters on input and 16B on output from a 552B-parameter backbone, an asymmetric split”

multimodal architecture 1

Joint vision-language pre-training from the start core

“visual and text embeddings trained jointly from the start of pre-training rather than added afterward”

agentic post-training 2

End-to-end project development core

“improve end-to-end project development—from game creation to 3D modeling”

Reinforcement learning with self-feedback used

“uses reinforcement learning with self-feedback to improve end-to-end project development”

KV cache management 1

Compressed KV caching used

“Compressed KV caching cuts cache memory to roughly a quarter of the previous Flash generation”

inference scheduling 1

Provider failover routing used

“we can recover by routing to another healthy provider”

agentic scaffolding 1

API request conversion, prompt encoding, and response parsing used

“It converts Messages, Chat Completions, and Responses API requests into the Conversation format, encodes them into DeepSeek V4 and V4.1…”

filed at the root 1

Python reference implementation for prompt encoding used

“The encoding folder contains a self-contained Python reference implementation (encoding.py) with test cases for multi-turn conversations…”

kernel & quantization library 1

deepseek-recipe optional

“For production use, we additionally release deepseek-recipe, a set of Rust libraries with Python bindings that provides the same prompt…”

unfiled 1

1 mentioned or evaluated
Local inference using converted model weights mentioned

“Please refer to the inference folder for instructions on weight conversion and running inference locally.”

Sources

The curated source list for this model. "not fetched" means the URL is recorded but its text was not read in the current run.

SourceKindPublisherStatus
api-docs.deepseek.com/news/news261017official blogDeepSeekread
huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/blob/main/DeepSeek_V41_T…technical reportDeepSeekread
huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017model cardDeepSeekread
openrouter.ai/deepseek/deepseek-v4.1-flash-1017vendor docsOpenRouterread
x.com/YingjunWu/article/2117387909183816165third party analysisYingjun Wu (Recurrence Research)not fetched
↑