28 open-weight models · 164 source documents · every claim quoted
What’s actually inside today’s open‑weight models.
We read the technical reports, model cards and repos behind the most-used open models and pulled out every method they describe — each one linked to the exact sentence that says so.
The consensus recipe
If you built a model from what the field agrees on, it would look like this.
The methods the most labs report adopting at each stage of the pipeline, one per sub-area. Numbers count labs, not models, so a lab that documents seven variants in one report still counts once — 13 labs in all.
Bar length is relative to Mixture of Experts, adopted by 13 of 13 labs. Adopted means a document states the model uses it, ships it as an option, or builds on it — not only that it discusses it.
Who uses what
One grid, 13 labs, the methods where they part ways.
Each column is one lab’s most thoroughly documented model. Rows are methods common enough to look for in a model’s documentation, picked where the labs’ answers differ most — the ones every lab adopts are in the recipe above.
A blank cell means the model’s documents don’t say — not that the model doesn’t use it.
A ✓ beside a cell means the model’s own source code shows the method, whatever its documents say; a ✕ that the code does not have it.
| Method | DeepSeek-V4.1-Flash | NVIDIA-Nemotron-3-Ultra-550B-A55B | Hy3 | GLM-5.2 | MiniMax-M3 | Gemma 4 31B | Inkling | Kimi K3 | Laguna-S-2.1 | MiMo-V2.6-Pro | Qwen3.8-Flash-Next | Step-3.7-Flash | gpt-oss-120b |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| model architecture | |||||||||||||
| Hybrid Attention | not in its documents | not in its documents | not in its documents | not in its documents | not in its documents | core | core | core | core | core | core | not in its documents | used |
| Multi-Token Prediction | not in its documents; in its code | core; in its code | core; not in its code | used | not in its documents; in its code | not in its documents; not in its code | not in its documents; in its code | not in its documents; not in its code | not in its documents; not in its code | core | core; in its code | optional; in its code | not in its documents; not in its code |
| Grouped-query attention | not in its documents; not in its code | not in its documents; in its code | core; in its code | not in its documents; not in its code | core; in its code | not in its documents; in its code | used; in its code | not in its documents; not in its code | core; in its code | not in its documents; in its code | not in its documents; in its code | not in its documents; in its code | used; in its code |
| optimization | |||||||||||||
| Muon | used | not in its documents | not in its documents | used | not in its documents | not in its documents | not in its documents | used | core | not in its documents | core | not in its documents | not in its documents |
| Expert Parallelism | not in its documents | used | not in its documents | used | not in its documents | not in its documents | not in its documents | used | not in its documents | not in its documents | not in its documents | used | not in its documents |
| Warmup-Stable-Decay | not in its documents | used | not in its documents | not in its documents | not in its documents | not in its documents | not in its documents | not used | used | not in its documents | not in its documents | not in its documents | not in its documents |
| post-training | |||||||||||||
| Asynchronous reinforcement learning | not in its documents | used | not in its documents | used | not in its documents | not in its documents | not in its documents | not in its documents | not in its documents | core | not in its documents | not in its documents | not in its documents |
| inference & serving | |||||||||||||
| Speculative decoding | used | default | core | used | not in its documents | core | not in its documents | not in its documents | not in its documents | not in its documents | not in its documents | optional | not in its documents |
| FP8 KV-cache quantization | not in its documents | used | optional | not in its documents | not in its documents | not in its documents | not in its documents | not in its documents | used | not in its documents | not in its documents | used | not in its documents |
| CUDA Graph | not in its documents | not in its documents | not in its documents | not in its documents | used | not in its documents | not in its documents | not in its documents | not in its documents | not in its documents | used | not in its documents | used |
Most widely attested
Ranked by how many documents describe the method, not by how often a single document repeats it.
| Technique | Area | Sources | Quotes |
|---|---|---|---|
| Mixture of Experts | model architecture | 88 | 101 |
| Multi-Token Prediction | model architecture | 30 | 40 |
| Configurable reasoning effort | inference & serving | 33 | 34 |
| Hybrid Attention | model architecture | 27 | 29 |
| Speculative decoding | inference & serving | 22 | 25 |
| Multi-Token Prediction | inference & serving | 20 | 20 |
| YaRN | model architecture | 12 | 19 |
| DeepSeek Sparse Attention | model architecture | 15 | 17 |
| Group Relative Policy Optimization | post-training | 15 | 17 |
| Multi-Teacher On-Policy Distillation | post-training | 12 | 17 |
| Supervised fine-tuning | post-training | 14 | 17 |
| FP8 KV-cache quantization | inference & serving | 13 | 15 |
Areas
The nine roots of the curated method taxonomy. A method is filed into a path that already exists; what fits nowhere becomes a proposal, listed on About.
model architecture
312 techniques
Mixture of Experts, Multi-Token Prediction, Hybrid Attention, YaRN, DeepSeek Sparse Attention
training objective
29 techniques
Multi-Token Prediction, Multi-Token Prediction Boosting, KL alignment loss, Logit matching, Sample-level attention masking
optimization
155 techniques
Expert Parallelism, Muon, NVFP4, Quantization-Aware Training, Tensor Parallelism
data curation
160 techniques
Agentic data synthesis pipeline, CSAM filtering, Knowledge distillation for synthetic data, Sensitive data filtering, Automated data synthesis and environment-construction pipelines
post-training
267 techniques
Group Relative Policy Optimization, Multi-Teacher On-Policy Distillation, Supervised fine-tuning, Reinforcement Learning, On-Policy Distillation
inference & serving
375 techniques
Configurable reasoning effort, Speculative decoding, Multi-Token Prediction, FP8 KV-cache quantization, FP8
software implementation
79 techniques
vLLM, Automatic provider failover, Hugging Face Transformers, OpenCode, SGLang
Models covered
| Rank | Model | Creator | Techniques | Sources |
|---|---|---|---|---|
| 1 | GLM-5.3-Flash | Z.ai (Zhipu AI) | 33 | 5 |
| 2 | DeepSeek-V4.1-Flash | DeepSeek | 201 | 4 |
| 3 | Hy4-preview | Tencent (Hunyuan) | 37 | 7 |
| 4 | DeepSeek-V4-Flash-0731 | DeepSeek | 17 | 4 |
| 5 | NVIDIA-Nemotron-3-Ultra-550B-A55B | NVIDIA | 208 | 7 |
| 6 | MiMo-V2.6-Flash | Xiaomi | 136 | 5 |
| 7 | DeepSeek-V4-Flash | DeepSeek | 82 | 4 |
| 8 | MiMo-V2.5 | Xiaomi | 77 | 8 |
| 9 | GLM-5.3 | Z.ai (Zhipu AI) | 46 | 6 |
| 10 | Hy3 | Tencent (Hunyuan) | 38 | 7 |
| 11 | GLM-5.2 | Z.ai (Zhipu AI) | 72 | 5 |
| 12 | MiniMax-M3 | MiniMax | 83 | 7 |
| — | DeepSeek-V3.2 | DeepSeek | 68 | 4 |
| — | DeepSeek-V4-Flash-Vision-Exp | DeepSeek | 25 | 3 |
| — | DeepSeek-V4-Pro | DeepSeek | 98 | 5 |
| — | DeepSeek-V4-Pro-0813 | DeepSeek | 13 | 3 |
| — | Gemma 4 31B | Google DeepMind | 78 | 10 |
| — | Inkling | Thinking Machines Lab | 73 | 11 |
| — | Kimi K3 | Moonshot AI | 183 | 10 |
| — | Laguna-S-2.1 | Poolside | 163 | 8 |
| — | MiMo-V2.5-Pro | Xiaomi | 41 | 6 |
| — | MiMo-V2.6-Pro | Xiaomi | 143 | 5 |
| — | NVIDIA-Nemotron-3.5-Lightning-30B-A3B | NVIDIA | 73 | 7 |
| — | Qwen3.5-397B-A17B | Alibaba (Qwen) | 98 | 8 |
| — | Qwen3.6-35B-A3B | Alibaba (Qwen) | 46 | 6 |
| — | Qwen3.8-Flash-Next | Alibaba (Qwen) | 126 | 7 |
| — | Step-3.7-Flash | StepFun | 42 | 6 |
| — | gpt-oss-120b | OpenAI | 86 | 6 |
Rank is the model's position among open-weight models in OpenRouter's token ranking for the week of 2026-09-21 (a partial week), the same week for every model; — means outside that week's top 12.
Extraction last run 2026-09-25.