Qwen

9 documents read from this publisher.

On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability

technical report · 2026-08-31 · 81 techniques

Mixture of Experts Gated DeltaNet Muon Gated Residual Manifold-Constrained Hyper-Connections N-gram embedding host-memory offload and prefetch N-gram embedding lookup Qwen Sparse Attention Batch-size warmup Updated hyperparameter scaling law CUDA Graph GatedNorm Split fused gradients before orthogonalization Hyper-Connections Independent Per-Branch Normalization Nesterov momentum Per-Branch Scalar Write Gate QK-Clip Reuse QSA index selection across speculative decoding steps Activation or logit clipping Adam without weight decay for the N-gram embedding table AdamW for attention and GDN output gates AdamW for gated-residual low-rank projections AdamW for input embeddings and output head AdamW for the MoE router All-branch Elementwise Read Gate Asynchronous Micro-Group pipeline Average pooling Batch-size warmup with adjusted peak learning rate Batch-size warmup with constant-batch optimum peak learning rate Block-causal scoring Bounded Positive Gates Canzona Contextual gating for N-gram embedding injection CUDA graph capture of the optimizer step Data-Dependent Residual Read and Write Operators Dense-attention distillation Eight-step Newton–Schulz iteration Elevated constant-learning-rate training stability stress test Fixed elevated learning rate

Qwen/Qwen3.8-Flash-Next · Hugging Face

first party release · model card · 2026-08-26 · 11 techniques

YaRN Default thinking mode Gated Residual Preserved thinking history mode N-gram embedding lookup Qwen Sparse Attention Multi-Token Prediction Category-specific assignment of Muon and AdamW Training without batch-size warmup Gated DeltaNet and Qwen Sparse Attention Increase video preprocessing longest-edge limit for higher frame-rate sampling

Qwen/Qwen3.6-27B · Hugging Face

first party release · model card · 2026-04-22 · 7 techniques

YaRN Default thinking mode Presence Penalty Prompt-based output standardization Preserve thinking Task-specific sampling parameters Adequate output length recommendation

Qwen/Qwen3.6-35B-A3B · Hugging Face

first party release · model card · 2026-04-22 · 15 techniques

Mixture of Experts YaRN Gated DeltaNet Presence Penalty Prompt-based output standardization Interleaved thinking between tool calls Thinking mode selection Gated Attention Language-model-only serving mode RoPE scaling MCP tool configuration Preserve thinking Dedicated serving engines Output length recommendation Video sampling parameters

Qwen/Qwen3.5-122B-A10B · Hugging Face

first party release · model card · 2026-03-09 · 27 techniques

YaRN Default thinking mode Excluding prior thinking from conversation history Disabling reasoning via chat-template configuration Early fusion multimodal training Presence Penalty Prompt-based output standardization NEXTN speculative decoding Asynchronous RL frameworks for large-scale agent scaffolds and environment orchestration 8 Routed + 1 Shared Experts Activated per Token Gated DeltaNet–sparse MoE hybrid MCP tool configuration Multi-step MTP training Qwen3 soft thinking switch Task-specific sampling parameters Adequate max output length setting (32,768 default; 81,920 for hard benchmarks)Gated DeltaNet with reduced KV-head configuration JSON-structured answer-format prompt Longest_edge video-preprocessing configuration for high frame-rate sampling of hour-scale videos Mode- and task-specific sampling parameter configuration Progressively Complex Task Distributions Scalable reinforcement learning across million-agent environments Serving Qwen3.5 with Hugging Face Transformers server Serving Qwen3.5 with KTransformers Serving Qwen3.5 with SGLang Serving Qwen3.5 with vLLM Step-by-step reasoning with boxed-answer prompt for math benchmarks

Qwen/Qwen3.5-397B-A17B · Hugging Face

first party release · model card · 2026-03-09 · 26 techniques

Multi-Token Prediction YaRN Gated DeltaNet Default thinking mode Discard-all context management Excluding prior thinking from conversation history Early fusion multimodal training Prompt-based output standardization Asynchronous RL frameworks for large-scale agent scaffolds and environment orchestration Gated Attention RoPE scaling Scalable RL at Agent Scale 10 Routed + 1 Shared Experts Activated per Token Context folding Hybrid Mamba-Transformer Mixture-of-Experts Layer Layout Multi-step MTP training Qwen-Agent Qwen3 soft thinking switch Tensor parallelism (degree 8)adequate output length (32,768 tokens default, 81,920 for hard benchmarks)configurable video frame sampling with fps and do_sample_frames hybrid Gated DeltaNet + sparse MoE architecture Qwen3 Coder tool-call parser recommended sampling parameters for thinking and non-thinking modes setting video preprocessor longest_edge to 469,762,048 for hour-scale video understanding vLLM language-model-only mode

Qwen/Qwen3.5-35B-A3B · Hugging Face

first party release · model card · 2026-03-09 · 24 techniques

Multi-Token Prediction YaRN Gated DeltaNet Default thinking mode Excluding prior thinking from conversation history Disabling reasoning via chat-template configuration Early fusion multimodal training Presence Penalty Prompt-based output standardization NEXTN speculative decoding Asynchronous RL frameworks for large-scale agent scaffolds and environment orchestration Gated Attention Language-model-only serving mode RoPE scaling 8 Routed + 1 Shared Experts Activated per Token Context folding Gated DeltaNet–sparse MoE hybrid Hybrid Mamba-Transformer Mixture-of-Experts Layer Layout Qwen-Agent Task- and mode-specific sampling parameter recommendations Configurable video frame sampling (fps/do_sample_frames) during inference Scalable RL with progressively complex task distributions in million-agent environments Task-appropriate maximum output length Video preprocessor longest-edge configuration

Qwen3.6

first party release · code repo · 5 techniques

Preserved thinking history mode Early fusion multimodal training Scalable RL at Agent Scale Asynchronous RL training infrastructure with near-100% multimodal efficiency Gated Delta Networks with sparse Mixture-of-Experts

Qwen3.8-Flash-Next

first party release · code repo · 17 techniques

Mixture of Experts Gated DeltaNet Muon Gated Residual N-gram embedding host-memory offload and prefetch N-gram embedding lookup Qwen Sparse Attention Category-specific assignment of Muon and AdamW Updated hyperparameter scaling law Split fused gradients before orthogonalization Gated DeltaNet and Qwen Sparse Attention hybrid architecture Tensor parallelism (degree 4)Compressed lightweight indexer Continuous batching Dynamic Gating of Residual Reads and Writes Four-Branch Residual Stream Muon orthogonalization accuracy refinement