Open Models Almanac
ModelsQwen3.8-Flash-Next

Alibaba (Qwen)

Qwen3.8-Flash-Next

What Qwen3.8-Flash-Next’s own documents say it is built from — every method with the sentence that describes it.

Parameters
125B / 6B active
Context
256K
Attention
linear hybrid3:1 Gated DeltaNet : Qwen Sparse Attention (block-level, lightweight indexer)
Experts
512 routed + 1 shared, top-10; plus 51B n-gram embedding params
Input
text, image, video
Licence
Qwen Community License 1.0
Released
2026-08-26

Against the consensus recipe

Its documents state 5 of the 18 methods the field agrees on.

The recipe from the overview: the methods the most labs adopt, per stage. A missing one is something its documents do not say, not something the model lacks.
  1. Data curation

  2. Training objective

  3. Model architecture

  4. Optimization

  5. Post-training

  6. Inference & serving

Checked in its code

Its code shows 11 of the 19 architecture features checked — 7 of them are not stated in its documents.

Sources say what a lab chose to describe; the modeling code says what the checkpoint runs. Each feature below was put to a code verifier, and only the config values and source lines it cited that were found in the files are shown, linked to the line.

Read from Hugging Face Transformers (modeling_qwen4_exp.py, configuration_qwen4_exp.py) and config.json at revision 812d96e10a, by anthropic/claude-sonnet-5 on 2026-11-07.

FeatureCodeIts documentsVerdict
Grouped-query attention code only in its codenot stated

Full-attention (indexed_attention) layers project 24 query heads but only 2 key/value heads, repeated via repeat_kv with…

show evidence

Full-attention (indexed_attention) layers project 24 query heads but only 2 key/value heads, repeated via repeat_kv with num_key_value_groups.

text_config.num_attention_heads = 24text_config.num_key_value_heads = 2
self.num_key_value_groups = config.num_attention_heads // config.num_key_value_heads
modeling_qwen4_exp.py · L813
self.k_proj = nn.Linear(
            config.hidden_size, config.num_key_value_heads * self.head_dim, bias=config.attention_bias
        )
modeling_qwen4_exp.py · L820–822
Multi-head latent attention not in its codenot stated

No kv_lora_rank/q_lora_rank or latent KV down/up-projection is present; attention uses standard per-head k_proj/v_proj instead.

Sliding-window attention not in its codenot stated

There is no sliding_window config key or windowed mask logic in the attention/masking code for this model; layer_types only distinguishes…

show the whole verdict

There is no sliding_window config key or windowed mask logic in the attention/masking code for this model; layer_types only distinguishes linear_attention and indexed_attention (full attention with an indexer), with no local window.

Interleaved sliding-window and global attention not in its codenot stated

layer_types interleaves linear_attention with full/indexed attention, not sliding-window with global attention, so there is no…

show evidence

layer_types interleaves linear_attention with full/indexed attention, not sliding-window with global attention, so there is no sliding-window/global interleaving.

text_config.layer_types = ["linear_attention", "linear_attention", "linear_attention", "full_attention"…
self.layer_types = [
                "linear_attention" if (i + 1) % interval_pattern else "indexed_attention"
                for i in range(self.num_hidden_layers)
            ]
configuration_qwen4_exp.py · L175–178
Indexer-selected sparse attention (DeepSeek Sparse Attention) code only in its codenot stated

A Qwen4ExpTextQSAIndexer scores compressed key blocks per query head and selects a top-k token budget, masking attention to only those…

show evidence

A Qwen4ExpTextQSAIndexer scores compressed key blocks per query head and selects a top-k token budget, masking attention to only those tokens, enabled via indexer_n_heads/indexer_budget/indexer_compress_ratio in the config.

text_config.indexer_n_heads = 4text_config.indexer_budget = 2048text_config.indexer_compress_ratio = 4
self.index_n_heads = config.indexer_n_heads
        self.index_kv_heads = config.indexer_kv_heads
        self.index_head_dim = config.indexer_head_dim
        self.token_budget = config.indexer_budget
        self.compress_ratio = config.indexer_compress_ratio
        self.block_topk = self.token_budget // self.compress_ratio
modeling_qwen4_exp.py · L669–674
selected_block_indices = scores.topk(min(self.block_topk, num_complete_blocks), dim=0).indices
modeling_qwen4_exp.py · L747
Linear-attention or state-space layers alongside full attention code only in its codenot stated

layer_types mixes linear_attention (Gated DeltaNet) layers with full/indexed-attention softmax layers in a fixed pattern.

show evidencetext_config.layer_types = ["linear_attention", "linear_attention", "linear_attention", "full_attention"…
if self.layer_type == "linear_attention":
            self.linear_attn = Qwen4ExpTextGatedDeltaNet(config, layer_idx)
        else:
            self.self_attn = Qwen4ExpTextAttention(config, layer_idx)
modeling_qwen4_exp.py · L1251–1254
Gated DeltaNet layers in its codeadopted

Qwen4ExpTextGatedDeltaNet implements the gated delta rule with a decay gate (A_log/dt_bias) and delta-rule state update, used on…

show evidence

Qwen4ExpTextGatedDeltaNet implements the gated delta rule with a decay gate (A_log/dt_bias) and delta-rule state update, used on linear_attention layers.

text_config.linear_num_key_heads = 16text_config.linear_conv_kernel_dim = 4
g = -self.A_log.float().exp() * F.softplus(a.float() + self.dt_bias)
modeling_qwen4_exp.py · L572
last_recurrent_state = last_recurrent_state * chunk_decay[:, :, i] + key[:, :, i].transpose(-1, -2) @ v_new
modeling_qwen4_exp.py · L376
Mamba-2 layers not in its codenot stated

No Mamba-2/SSD selective state-space block is implemented; the recurrent mixer present is the gated delta rule (GatedDeltaNet), not Mamba-2.

Learnable attention sink not in its codenot stated

No per-head learned sink logit is added to the attention softmax anywhere in the code.

Gated attention output code only in its codenot stated

Attention output is multiplied by a sigmoid gate derived from a chunk of the q_proj output before the output projection.

show evidence
attn_output = attn_output.reshape(*input_shape, -1).contiguous()
        attn_output = attn_output * torch.sigmoid(gate)
modeling_qwen4_exp.py · L884–885
QK normalization code only in its codenot stated

Queries and keys are each passed through an RMSNorm (q_norm/k_norm) per head before RoPE and the attention dot product.

show evidence
query_states = self.q_norm(query_states.view(hidden_shape)).transpose(1, 2)
        key_states = self.k_norm(self.k_proj(hidden_states).view(hidden_shape)).transpose(1, 2)
modeling_qwen4_exp.py · L859–860
Partial RoPE in its codeadopted

partial_rotary_factor of 0.25 restricts RoPE to a quarter of the head dimension, with the rest (q_nope/k_nope) concatenated back unrotated.

show evidencetext_config.partial_rotary_factor = 0.25text_config.rope_parameters.partial_rotary_factor = 0.25
q_rope, q_nope = q[..., :rotary_dim], q[..., rotary_dim:]
    # Apply rotary embeddings on the first half or full tensor
    q_rope = (q_rope * cos) + (rotate_half(q_rope) * sin)
    # Concatenate back to full shape
    q_rotated = torch.cat([q_rope, q_nope], dim=-1)
modeling_qwen4_exp.py · L648–652
YaRN RoPE scaling documents disagree not in its codeadopted

rope_parameters.rope_type is set to "default", not YaRN, so no YaRN scaling is applied.

show evidencetext_config.rope_parameters.rope_type = "default"
Layers without positional encoding (NoPE) not in its codenot stated

All softmax-attention (indexed_attention/full_attention) layers apply the same partial RoPE via apply_rotary_pos_emb; there is no…

show the whole verdict

All softmax-attention (indexed_attention/full_attention) layers apply the same partial RoPE via apply_rotary_pos_emb; there is no per-layer toggle disabling RoPE entirely on some attention layers (linear_attention layers are not softmax attention and don't count).

Mixture of experts in its codeadopted

The MLP block routes each token to num_experts_per_tok of num_experts experts via a learned TopKRouter.

show evidencetext_config.num_experts = 512text_config.num_experts_per_tok = 10
router_top_value, router_indices = torch.topk(router_probs, self.top_k, dim=-1)  # (seq_len, top_k)
modeling_qwen4_exp.py · L960
Shared expert code only in its codenot stated

Each MoE block has a shared_expert MLP with nonzero shared_expert_intermediate_size applied to every token and gated before being added to…

show evidence

Each MoE block has a shared_expert MLP with nonzero shared_expert_intermediate_size applied to every token and gated before being added to routed expert output.

text_config.shared_expert_intermediate_size = 640
self.shared_expert = Qwen4ExpTextMLP(config, intermediate_size=config.shared_expert_intermediate_size)
        self.shared_expert_gate = torch.nn.Linear(config.hidden_size, 1, bias=False)
modeling_qwen4_exp.py · L973–974
Auxiliary-loss-free load balancing (selection bias) not in its codenot stated

The router in Qwen4ExpTextTopKRouter computes plain softmax + topk with no per-expert correction bias (e_score_correction_bias) added to…

show evidence

The router in Qwen4ExpTextTopKRouter computes plain softmax + topk with no per-expert correction bias (e_score_correction_bias) added to the scores.

router_probs = torch.nn.functional.softmax(router_logits, dtype=torch.float, dim=-1)
        router_top_value, router_indices = torch.topk(router_probs, self.top_k, dim=-1)  # (seq_len, top_k)
modeling_qwen4_exp.py · L959–960
Multi-token prediction layers in its codeadopted

The config declares an mtp block with num_hidden_layers=1 (also mtp_num_hidden_layers=1), i.e. extra next-token prediction layer(s) are…

show evidence

The config declares an mtp block with num_hidden_layers=1 (also mtp_num_hidden_layers=1), i.e. extra next-token prediction layer(s) are declared in config.

text_config.mtp_num_hidden_layers = 1text_config.mtp.num_hidden_layers = 1
SwiGLU feed-forward code only in its codenot stated

Both the dense MLP and expert MLPs compute silu(gate_proj(x)) * up_proj(x), i.e. SwiGLU, matching hidden_act=silu.

show evidencetext_config.hidden_act = "silu"
down_proj = self.down_proj(self.act_fn(self.gate_proj(x)) * self.up_proj(x))
modeling_qwen4_exp.py · L903
current_hidden_states = self.act_fn(gate) * up
modeling_qwen4_exp.py · L939

token mixer 14

sparse attention 11

Qwen Sparse Attention core

“it aggregates the sequence into micro-blocks, estimates importance at the block level, and then selects the most relevant regions”

Fixed-budget sparse-attention selection core

“Attention cost stops growing with context length once the budget binds”

QSA micro-block compression at ratio 4 core

“uses compressed QSA with a compression ratio of 4, or c4.”

Joint backbone and indexer training under sparse attention used

“During the final stage of CPT, QSA is enabled and the backbone and indexer are jointly trained for 8,000 steps”

block selection 2

Top-512 block selection core

“QSA keeps the best 512 blocks, expands them back to 2048 logical token positions”

Top-scoring compressed-block selection used

“each query selects the highest-scoring compressed blocks”

sparse attention indexer 5

Reuse QSA index selection across speculative decoding steps core

“So the draft decode steps stop running the indexer altogether.”

Compressed lightweight indexer core

“uses a compressed lightweight indexer to select the important context at micro-block granularity”

MQA indexer core

“The selection machinery is small and explicit: an MQA indexer with 4 query heads and a single shared key head, head dimension 128…”

Average pooling used

“keys are partitioned into non-overlapping blocks of tokens and compressed by average pooling”

Block-causal scoring used

“Block-level importance scores are then obtained through block-causal scoring.”

linear attention & state space 2

gated delta network 2

Gated DeltaNet core

“GDN compresses the prefix into a fixed-size recurrent state and updates that state according to the current content”

Bounded sigmoid output gate used

“Unlike the original GDN, which uses a SiLU output gate, we use the bounded sigmoid gate”

hybrid layer stacking 1

Gated DeltaNet and full attention core

“The 3:1 interleave of Gated DeltaNet to full-attention layers is the same ratio Qwen3.5 uses”

channel mixer 3

Contextual gating for N-gram embedding injection used

“The embeddings are further injected into residual stream through contextual gating mechanism”

mixture of experts 2

Mixture of experts core

“Qwen3.8-Flash-Next is a sparse mixture-of-experts model”

shared experts 1

MoE with 10 routed and 1 shared expert activated per token core

“MoE experts: 512 experts, 10 routed + 1 shared activated per token”

positional encoding 3

Static YaRN used

“All the notable open-source frameworks implement static YaRN, which means the scaling factor remains constant regardless of input length”

Partial rotary position embeddings used

“partial RoPE is applied to 64 of the 128 dimensions in each indexer head”

YaRN optional

“For long-horizon tasks where the total length (including both input and output) exceeds this limit, we recommend using RoPE scaling…”

normalization & residual 14

Gated Residual core

“the gate decides how that capacity is spent, while also supplying the rescaling that keeps training stable”

Per-branch scalar residual write gate core

“The block output is written to every branch through one data-dependent scalar per branch”

All-branch, elementwise read gating in Gated Residual core

“It then predicts elementwise gating scores per branch and channel from all branches”

Dynamic gating of residual reads and writes core

“controls reads and writes with a dynamic gate”

Element-wise data-dependent residual read gate core

“The read gate is element-wise and data-dependent”

Four-branch residual stream core

“widens the residual stream into 4 branches”

Four-stream Hyper-Connection Combine update core

“Combine computes four injection coefficients and applies a residual update to the four streams.”

GatedNorm core

“the multiplicative gate supplies the necessary rescaling directly, keeping the training stable”

Independent per-branch normalization in Gated Residual core

“GR first normalizes each branch independently,”

Low-rank gated Mix for Hyper-Connection residual streams core

“Mix uses a low-rank projection to generate element-wise gates and reduce the four residual streams into one hidden state.”

Widened residual stream core

“The residual stream is widened to four branches”

Bounded positive gates used

“A sigmoid gate is better than in both loss and training stability.”

Data-dependent residual read and write operators used

“Making and data-dependent reduces loss by 0.002 over the static variant”

Zero-centered RMSNorm used

“we adopt zero-centered RMSNorm to constrain the growth of RMSNorm weights”

prediction head 1

Multi-Token Prediction core

“MTP: 1 layer, trained with multi-steps”

context capacity 7

N-gram embedding lookup memory core

“short n-grams ending at each token serve as keys into embedding tables, and the retrieved vectors augment the corresponding token…”

Host-memory-prefetched N-gram embedding tables core

“Capacity is further added outside the backbone by a single n-gram embedding layer whose tables are prefetched from host memory”

N-gram embedding table for parameter scaling core

“Twenty million bigram and trigram embeddings are applied at layer 2, contributing 51B parameters — more than 40% of the checkpoint's…”

Single-layer N-gram embedding placement core

“A single N-gram embedding layer is sufficient. ... We place it at Layer 2”

Host-memory offload for N-gram embeddings used

“The N-gram Embedding can reside entirely in host memory to save GPU memory”

Asynchronous prefetching of offloaded N-gram embeddings used

“lookup positions are computed in advance and asynchronously prefetched, so it never permanently occupies GPU memory.”

Multi-head hashing for N-gram embedding lookup used

“We use multi-head hashing for embedding lookup”

5 more mentioned or evaluated
Manifold-Constrained Hyper-Connections evaluated

normalization & residual “mHC (Xie et al., 2025) uses a sigmoid and additionally constrains to a manifold of doubly stochastic matrices.”

Hyper-Connections evaluated

normalization & residual “HC generalizes Eq. (21) and Eq. (22) into three learnable operators.”

Frequency-based partitioning of N-gram embedding slots evaluated

mixture of experts “frequency-based partitioning of embedding slots. Despite these efforts, we observed no consistent performance gains in our training recipe.”

Simplified AltUp variant evaluated

normalization & residual “We first study how much of the reported gain comes from width alone, using a simplified variant of AltUp”

Token normalization for N-gram vocabulary compression evaluated

context capacity “including but not limited to token normalization for vocabulary compression ... Despite these efforts, we observed no consistent…”

Not used 3

SwiGLU-clip not used

dense feed-forward network “This allows stable training without explicit activation control such as qk-clip and SwiGLU-clip”

QK-Clip not used

normalization & residual “This allows stable training without explicit activation control such as qk-clip”

Sparse gated residual writes not used

normalization & residual “each block reads only the two branches with the highest gate values instead of all. Pre-training loss and benchmarks were almost…”

multi-token prediction objective 1

Multi-Token Prediction used

“MTP: 1 layer, trained with multi-steps”

distillation objective 1

Dense-attention distillation to initialize the QSA indexer used

“Stage 1: Dense Distillation. We first distill the full-sequence attention distribution of the backbone into the indexer.”

filed at the root 2

Hyperparameter scaling-law fitting core

“Therefore, we develop an updated hyperparameter scaling law.”

Phase-by-phase training cost accounting used

“Phase-by-phase cost accounting directed the sparse-attention design into the indexer”

optimizer 15

Muon core

“We use Muon as the main optimizer”

AdamW core

“Muon exacerbates early-training fluctuations and destabilizes the router. ... Therefore, we use AdamW for the router.”

Category-specific optimizer assignment core

“The Muon and AdamW optimizers are applied to specific weight categories to maximize efficiency.”

Splitting fused parameters before orthogonalization core

“We therefore split the fused gradient before orthogonalization, run NS on each sub-matrix independently, and gather the results back into…”

Restricting Muon to two-dimensional linear-map weights core

“We apply Muon to the two-dimensional weights that genuinely act as linear maps”

Newton–Schulz orthogonalization default

“We set the Newton–Schulz iteration to 8 steps, favouring the additional stability under stress.”

Adam used

“the -gram embedding table runs on Adam with weight decay disabled.”

Nesterov momentum used

“The NS iteration is applied to a Nesterov-accelerated momentum”

Asynchronous Micro-Group pipeline used

“an asynchronous Micro-Group pipeline reconstructs each Muon-owned matrix via fused All-to-All across TP ranks.”

Canzona used

“We developed Canzona, which decouples logical optimizer assignment from physical parameter layout”

CUDA graph capture of the optimizer step used

“We capture the whole step in a CUDA graph to remove this overhead.”

Muon orthogonalization accuracy refinement used

“refined around orthogonalization accuracy”

Per-head splitting of attention and GDN input projections used

“The and GDN input projections are split at per-head granularity, which improves both loss and downstream benchmarks.”

Polar Express coefficient schedule used

“we adopt the per-step coefficient schedule of the Polar Express method”

Splitting SwiGLU fc1 into gate and up halves used

“The fc1 is split into its gate and up halves; the loss is essentially unchanged while benchmarks improve slightly.”

learning-rate schedule 3

No batch-size warmup used

“we eliminate traditional batch-size warmups and start directly at the target batch size”

Fixed elevated learning rate used

“we hold the learning rate constant at a multiple of its optimal value, bypassing the standard decay schedule to simulate the prolonged…”

Scaling-law fit for learning-rate and batch-size selection used

“Downstream benchmark results demonstrate the robust convergence achieved by our new scaling-law fit”

training precision 1

FP8 residual-state storage used

“Storing the branches in FP8 halves the bytes moved for the residual state relative to BF16, with almost no loss in quality.”

training stability 2

Simulate large-scale training instability by raising the learning rate used

“large-scale instabilities can be reproduced in small models by raising the learning rate”

Training stability stress testing used

“we design a set of stress tests that amplify the relevant stress within a smaller budget”

training parallelism 2

Tensor and expert parallelism used

“Use TEP8 with the Triton MoE backend on Hopper”

NS-FLOP-balanced static parameter partitioning used

“an -balanced static partitioner reassigns whole parameters (no cut inside a tensor) to equalize estimated NS FLOPs across DP ranks”

1 more mentioned or evaluated
Batch-size warmup evaluated

learning-rate schedule “the first maintains the peak learning rate of the constant-batch optimum, making the ramp its only difference”

Not used 1

Activation or logit clipping not used

training stability “without relying on explicit clipping methods such as qk-clip or SwiGLU-clip”

decoding strategy 2

NEXTN speculative decoding used

“the model ships a 1-layer MTP head trained with multiple steps, exposed by SGLang as native NEXTN speculative decoding.”

Multi-Token Prediction optional

“the built-in Multi-Token Prediction module supports speculative decoding.”

reasoning control 1

Reasoning enabled by default default

“Qwen3.8-Flash-Next models operate in thinking mode by default”

KV cache management 1

Overlapped host-memory prefetching used

“We place it at Layer 2, allowing host-memory prefetching to overlap with the computation of the first layer.”

inference quantization 1

Fine-grained FP8 quantization optional

“an FP8 checkpoint that uses fine-grained quantization with a block size of 128”

serving parallelism 2

Tensor parallelism default

“vllm serve Qwen/Qwen3.8-Flash-Next-FP8 \ --tensor-parallel-size 4”

Expert parallelism optional

“Enable expert parallelism with --enable-expert-parallel to improve throughput.”

inference scheduling 1

Continuous batching optional

“transformers serve Qwen/Qwen3.8-Flash-Next --port 8000 --continuous-batching”

inference kernel 7

Shape-aware kernel dispatch for Hyper-Connection core

“Shape-aware dispatch lets HC use the appropriate execution path for both low-latency decode and large-scale prefill.”

Sparse pinned-host offload default

“SGLang keeps each rank's vocabulary-parallel table shard in pinned host memory and gathers the selected rows into a small BF16 GPU buffer…”

CUDA Graph capture and replay used

“the step is captured in a CUDA graph”

Fused QSA attention and KL-loss kernel used

“we implement a fused QSA kernel that jointly computes sparse attention outputs and the KL loss without materializing intermediate results”

Fusing activation, gating, and reduction into GEMM epilogues used

“SiLU, Sigmoid, gating, and the final reduction are fused into the two GEMM epilogues, avoiding intermediate writes to global memory.”

Hidden-dimension split Combine kernel used

“At small M, this mapping exposes too few CTAs, so the M ≤ 32 path splits each row along the hidden dimension.”

Split-K CuTe GEMM used

“For M ≤ 16, we use the low-latency split-K CuTe GEMM from FlashInfer PR #4266.”

context management 1

Preserved thinking history default

“By default, Qwen3.8-Flash-Next retains thinking blocks from all historical messages, maintaining a complete reasoning trace across the…”

kernel & quantization library 1

FlashQLA used

“we optimize the GDN kernel with FlashQLA, a TileLang-based fused linear-attention kernel library”

filed at the root 2

Fixed MTP acceptance length used

“runs therefore apply a fixed acceptance length measured separately”

Joint validation protocol used

“Where pre-training metrics agreed but late-stage evaluation diverged, the joint protocol caught the discrepancy before it reached production”

unfiled 23

Gated DeltaNet and Qwen Sparse Attention hybrid architecture core

“out of every 4 layers, 3 GDN layers compress history into a fixed-size state, while the remaining layer performs precise retrieval over…”

Gated DeltaNet and global-attention hybrid core

“Token mixing uses a layer-wise hybrid of Gated DeltaNet (GDN) and global attention”

Gated DeltaNet and Qwen Sparse Attention hybrid core

“three of every four layers use Gated DeltaNet to compress history; the fourth uses Qwen Sparse Attention for precise long-range retrieval.”

IndexShare for multi-token prediction speculative decoding core

“the QSA top-k selection computed by the draft-extend pass over the tokens the target just accepted is held for the whole MTP iteration”

Micro-block compressed lightweight indexer core

“QSA employs a lightweight indexer that compresses the sequence into micro-block representations, estimates their importance, and selects…”

Omit residual branch-mixing operator default

“The efficiency advantage of GR is that removing eliminates a full read of the residual state per block”

One global-attention layer per four-layer GDN hybrid block default

“We place one such full-attention layer in every four layers, with the other three layers using GDN.”

PLE CPU offload for N-gram embeddings default

“the recipe enables PLE CPU offload automatically on this hardware.”

Failover routing across providers used

“we can recover by routing to another healthy provider”

Fuse gated residual reads and writes into single kernels used

“the read of Eq. (30)–(32) and the write of Eq. (33)–(34) are each fused into a single kernel, with the group RMSNorm folded into the read”

Fused gated residual read and final normalization used

“Fusing the residual read and the final normalization before the LM head into one gated read operation further reduces the gradient norm”

Limit maximum concurrent sequences to 256 used

“Mamba-cache capacity error at startup: keep --max-num-seqs 256.”

Max-pool teacher attention distributions to block level used

“we apply max pooling to align this distribution with the block-level indexer scores”

Measure per-block maximum activation used

“and the per-block maximum activation”

Monitor pre-clip gradient norms and clipping-threshold crossings used

“the of the pre-clip gradient norm and the number of threshold crossings”

Offline weight reordering for local gate reduction used

“The up-projection weights are reordered offline so the four gate values for each output can be reduced locally inside a tile.”

Overlap pinned-host embedding gather with decoder computation used

“A dedicated CUDA stream overlaps the gather with the first decoder block.”

Request-filter-constrained provider routing used

“if your request filters allow it”

Rolling-median loss-spike measurement used

“loss spikes (steps exceeding a -step rolling median by more than )”

Increase video preprocessing longest-edge limit for higher frame-rate sampling optional

“It is recommended to set the longest_edge parameter in the video_preprocessor_config file to 469,762,048 (corresponding to 224k video…”

2 more mentioned or evaluated
Non-uniform embedding-slot allocation across N-gram orders evaluated

“including but not limited to token normalization for vocabulary compression ..., non-uniform allocation across N-gram orders, and…”

Tiered CPU and filesystem offloading evaluated

“Validation used TieringOffloadingSpec: 8 GiB total CPU tier, blocks_per_chunk: 1”

Not used 1

Read only the two highest-gated residual branches not used

“each block reads only the two branches with the highest gate values instead of all”

Sources

The curated source list for this model. "not fetched" means the URL is recorded but its text was not read in the current run.

SourceKindPublisherStatus
qwen.ai/blog?id=qwen3.8-flash-nextofficial blogQwennot fetched
arxiv.org/abs/2608.30320technical reportQwenread
huggingface.co/Qwen/Qwen3.8-Flash-Nextmodel cardQwenread
github.com/QwenLM/Qwen3.8-Flash-Nextcode repoQwenread
lmsys.org/blog/2026-08-26-qwen-flash-nextvendor docsLMSYSread
recipes.vllm.ai/Qwen/Qwen3.8-Flash-Nextvendor docsvLLMread
inferencex.semianalysis.com/model/qwen-3-8-flash-nextthird party analysisSemiAnalysisread
openrouter.ai/qwen/qwen3.8-flashvendor docsOpenRouterread
↑