SourcesQwen
publisher · 9 sources
Qwen
9 sources read from this publisher, newest first, with the methods each one describes.
On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability
technical report · 2026-08-31 · 74 methods
Mixture of expertsMuonGated DeltaNetAdamWManifold-Constrained Hyper-ConnectionsBatch-size warmupGated ResidualCategory-specific optimizer assignmentCUDA Graph capture and replayQwen Sparse AttentionN-gram embedding lookup memoryHyperparameter scaling-law fittingGatedNormPartial rotary position embeddingsQK-ClipSplitting fused parameters before orthogonalizationAdamHyper-ConnectionsIndependent per-branch normalization in Gated ResidualNesterov momentumNewton–Schulz orthogonalizationPer-branch scalar residual write gateReuse QSA index selection across speculative decoding stepsTraining stability stress testingActivation or logit clippingAll-branch, elementwise read gating in Gated ResidualAsynchronous Micro-Group pipelineAverage poolingBlock-causal scoringBounded positive gatesBounded sigmoid output gateCanzonaContextual gating for N-gram embedding injectionCUDA graph capture of the optimizer stepData-dependent residual read and write operatorsDense-attention distillation to initialize the QSA indexerFixed elevated learning rateFlashQLAFP8 residual-state storageFrequency-based partitioning of N-gram embedding slots
Qwen/Qwen3.8-Flash-Next · Hugging Face
first party release · model card · 2026-08-26 · 12 methods
YaRNGated DeltaNet and full attentionPreserved thinking historyGated ResidualMulti-Token PredictionCategory-specific optimizer assignmentQwen Sparse AttentionStatic YaRNN-gram embedding lookup memoryReasoning enabled by defaultNo batch-size warmupIncrease video preprocessing longest-edge limit for higher frame-rate sampling
Qwen/Qwen3.6-27B · Hugging Face
first party release · model card · 2026-04-22 · 7 methods
Qwen/Qwen3.6-35B-A3B · Hugging Face
first party release · model card · 2026-04-22 · 15 methods
Mixture of expertsThinking modeGated DeltaNetYaRNPreserved thinking historyPrompt-based output standardizationGated AttentionInterleaved thinkingLanguage-model-only serving modeMCP configuration for tool definitionPresence penalty adjustmentRoPE scalingDedicated serving enginesMultimodal processor video-sampling parametersOutput length recommendation
Qwen/Qwen3.5-122B-A10B · Hugging Face
first party release · model card · 2026-03-09 · 28 methods
Thinking modeYaRNMulti-Token PredictionExcluding prior thinking from multi-turn historyStatic YaRNDisable reasoning via enable_thinkingEarly-fusion multimodal trainingPrompt-based output standardizationNEXTN speculative decodingAsynchronous RL framework for large-scale agent scaffolds and environment orchestrationPresence penalty tuningTask-specific sampling-parameter presetsGated DeltaNet–sparse Mixture-of-Experts hybridMCP configuration for tool definitionMoE with 8 routed and 1 shared expert activated per tokenQwen3 soft thinking switchAdequate max output length setting (32,768 default; 81,920 for hard benchmarks)Gated attention with reduced KV headsJSON answer-format promptingLongest_edge video-preprocessing configuration for high frame-rate sampling of hour-scale videosMode- and task-specific sampling parameter configurationProgressively complex task distributionsScalable reinforcement learning across million-agent environmentsServing Qwen3.5 with Hugging Face Transformers serverServing Qwen3.5 with KTransformersServing Qwen3.5 with SGLangServing Qwen3.5 with vLLMStep-by-step reasoning with boxed-answer prompt for math benchmarks
Qwen/Qwen3.5-397B-A17B · Hugging Face
first party release · model card · 2026-03-09 · 26 methods
Multi-Token PredictionGated DeltaNetYaRNGated DeltaNet and full attentionDiscard-all context managementMulti-Token PredictionExcluding prior thinking from multi-turn historyStatic YaRNEarly-fusion multimodal trainingPrompt-based output standardizationReasoning enabled by defaultTensor parallelismAsynchronous RL framework for large-scale agent scaffolds and environment orchestrationGated AttentionScalable RL at Agent ScaleContext foldingMoE with 10 routed and 1 shared expert activated per tokenQwen-AgentQwen3 soft thinking switchRoPE scalingTool calling with the qwen3_coder parseradequate output length (32,768 tokens default, 81,920 for hard benchmarks)configurable video frame sampling with fps and do_sample_framesrecommended sampling parameters for thinking and non-thinking modessetting video preprocessor longest_edge to 469,762,048 for hour-scale video understandingvLLM language-model-only serving mode
Qwen/Qwen3.5-35B-A3B · Hugging Face
first party release · model card · 2026-03-09 · 26 methods
Thinking modeMulti-Token PredictionGated DeltaNetGated DeltaNet and full attentionExcluding prior thinking from multi-turn historyStatic YaRNDisable reasoning via enable_thinkingEarly-fusion multimodal trainingPrompt-based output standardizationNEXTN speculative decodingAsynchronous RL framework for large-scale agent scaffolds and environment orchestrationGated AttentionLanguage-model-only serving modePresence penalty tuningYaRN scaling-factor selectionContext foldingGated DeltaNet–sparse Mixture-of-Experts hybridMoE with 8 routed and 1 shared expert activated per tokenQwen-AgentTask- and mode-specific sampling parametersConfigurable video frame sampling (fps/do_sample_frames) during inferenceRoPE scaling for context extension (YaRN)Scalable RL with progressively complex task distributions in million-agent environmentsTask-appropriate maximum output lengthVideo preprocessor longest-edge configurationYaRN configuration
Qwen3.6
first party release · code repo · 5 methods
Qwen3.8-Flash-Next
first party release · code repo · 18 methods
Mixture of expertsMuonGated DeltaNetGated ResidualCategory-specific optimizer assignmentQwen Sparse AttentionN-gram embedding lookup memoryHyperparameter scaling-law fittingTensor parallelismHost-memory offload for N-gram embeddingsSplitting fused parameters before orthogonalizationAsynchronous prefetching of offloaded N-gram embeddingsGated DeltaNet and Qwen Sparse Attention hybrid architectureCompressed lightweight indexerContinuous batchingDynamic gating of residual reads and writesFour-branch residual streamMuon orthogonalization accuracy refinement