specific method · filed under model architecture
YaRN
A RoPE-scaling method used to extend context length; the evidence also describes static scaling and workload-based factor selection.
Also called Modifying the model configuration file, Static YaRN context length extension, static YaRN, static YaRN long-context extension, Long Context feature, Static YaRN scaling factor.
- sources
- 12
- models
- 4
- labs adopt it
- 2
- strongest
- used
How sources treat it
One count per evidence span, weakest treatment to strongest.
Documented in
Further reading
Picked by hand, not extracted: where to read more, not evidence for anything on this page.
In the code
Checked as “YaRN RoPE scaling” against each model’s own modeling code and config.json, independently of its documents. A model’s page shows the quoted lines.
| Model | Code | Its documents | What the code does |
|---|---|---|---|
| GLM-5.3-Flash | not in its code | not stated | There is no rope_scaling or rope_parameters key of type yarn in text_config, and no YaRN scaling code is applied to the language model (rope is disabled entirely, qk_rope_head_dim=0). |
| DeepSeek-V4.1-Flash | in its code | not stated | rope_scaling in the config sets rope_type=yarn with factor/beta_fast/beta_slow, and build_deepseek_v4_rope constructs the rotary embedding from this config for use in attention. attention.py#L399 |
| Hy4-preview | not in its code | not stated | rope_parameters.rope_type is "default" rather than "yarn", so the YaRN scaling code path (yarn_apply_mscale checking rope_type != default) is not triggered. modeling_hy_v4.py#L321 |
| DeepSeek-V4-Flash-0731 | in its code | not stated | rope_scaling type is yarn and is applied to the compress rope-type branch used by CSA/HCA layers via ROPE_INIT_FUNCTIONS lookup keyed by rope_type. modeling_deepseek_v4.py#L105 |
| NVIDIA-Nemotron-3-Ultra-550B-A55B | not in its code | not stated | No rope_scaling key is present in config.json, and no YaRN-specific scaling code path is invoked. |
| MiMo-V2.6-Flash | not in its code | not stated | rope_parameters/rope_type is set to "default" in this config, not YaRN, so no YaRN scaling is applied. |
| DeepSeek-V4-Flash | in its code | not stated | rope_scaling type is yarn with factor 16, applied to the compress-branch rope parameters used by CSA/HCA layers via ROPE_INIT_FUNCTIONS lookup. modeling_deepseek_v4.py#L105 |
| MiMo-V2.5 | not in its code | not stated | rope_scaling type is 'default', not YaRN, so no YaRN scaling is applied. |
| GLM-5.3 | not in its code | not stated | rope_parameters.rope_type is 'default', not 'yarn', so the yarn_apply_mscale/yarn_get_mscale code paths are not activated by this config. modeling_glm_moe_dsa.py#L299 |
| Hy3 | not in its code | not stated | rope_parameters.rope_type is set to 'default', not YaRN, so no rope scaling is applied |
| GLM-5.2 | not in its code | not stated | rope_parameters.rope_type is 'default' (not 'yarn'), so although the code has yarn_apply_mscale/yarn_get_mscale helpers, this config does not enable YaRN scaling. modeling_glm_moe_dsa.py#L299 |
| MiniMax-M3 | code unclear | not stated | rope_parameters is not set in the given config (it is null/None by default) so whether YaRN scaling type is configured cannot be determined from these files. |
| DeepSeek-V3.2 | in its code | not stated | rope_scaling type is set to yarn in the config and the rotary embedding/mscale code applies YaRN-specific frequency and magnitude scaling. modeling_deepseek_v32.py#L300 |
| DeepSeek-V4-Flash-Vision-Exp | in its code | not stated | rope_scaling type is yarn and is applied to the 'compress' rope-type branch used by CSA/HCA compressor layers via the standard YaRN init function. modeling_deepseek_v4.py#L105 |
| DeepSeek-V4-Pro | in its code | not stated | rope_scaling type is yarn and is applied to the compress rope-type branch used by CSA/HCA layers via ROPE_INIT_FUNCTIONS configuration_deepseek_v4.py#L318 |
| DeepSeek-V4-Pro-0813 | in its code | not stated | rope_scaling type is yarn in the config and is applied to the compress rope-type branch used by CSA/HCA layers via the ROPE_INIT_FUNCTIONS yarn initializer. modeling_deepseek_v4.py#L105 |
| Gemma 4 31B | not in its code | not stated | rope_parameters specify rope_type 'proportional' and 'default', not YaRN, and no YaRN scaling code path is triggered by this config. |
| Inkling | not in its code | not stated | No rope_scaling config key or YaRN-related code exists anywhere in this model; positional signal uses a learned relative-bias projection instead of RoPE. |
| Kimi K3 | code unclear | not stated | The config exposes a rope_scaling attribute in KimiLinearConfig defaulting to None and it is not present as a key in the given text_config, and no rotary_emb / RoPE application code is shown (self.rotary_emb is set to None and never used), so YaRN scaling cannot be confirmed as active. modeling_kimi_linear.py#L403 |
| Laguna-S-2.1 | in its code | not stated | The full_attention layer type sets rope_type to yarn with factor/beta parameters, and the rotary embedding module dispatches to the YaRN init function. modeling_laguna.py#L81 |
| MiMo-V2.5-Pro | not in its code | not stated | rope_scaling is not set in this config (no key present) so no YaRN scaling is applied; rotary embedding falls back to the default type. modeling_mimo_v2.py#L448 |
| MiMo-V2.6-Pro | not in its code | not stated | rope_parameters/rope_type is set to "default", not YaRN, so no YaRN scaling is applied. |
| NVIDIA-Nemotron-3.5-Lightning-30B-A3B | code unclear | not stated | The config has no rope_scaling key at all, and the attention forward pass shown does not call apply_rotary_pos_emb or reference rope_scaling, so YaRN cannot be confirmed or ruled out from these files. |
| Qwen3.5-397B-A17B | not in its code | used | rope_parameters.rope_type is set to "default", not "yarn", so no YaRN scaling is switched on for this config. modeling_qwen3_5_moe.py#L154 |
| Qwen3.6-35B-A3B | not in its code | optional | rope_parameters.rope_type is set to "default", not "yarn", so no YaRN scaling is applied. |
| Qwen3.8-Flash-Next | not in its code | used | rope_parameters.rope_type is set to "default", not YaRN, so no YaRN scaling is applied. |
| Step-3.7-Flash | not in its code | not stated | rope_scaling is present but its rope_type is 'llama3', not YaRN, so no YaRN-specific scaling code path is triggered for this config. |
| gpt-oss-120b | in its code | used | rope_scaling specifies rope_type yarn with factor 32 and beta parameters, applied via ROPE_INIT_FUNCTIONS. modeling_gpt_oss.py#L158 |
Evidence
19 spans quoted from the sources, strongest treatment first.
extend the context length of dense layers to tokens using YaRN
All the notable open-source frameworks implement static YaRN, which means the scaling factor remains constant regardless of input length
All the notable open-source frameworks implement static YaRN, which means the scaling factor remains constant regardless of input length
For long-horizon tasks where the total length (including both input and output) exceeds this limit, we recommend using RoPE scaling techniques to handle long texts effectively, e.g., YaRN.
All the notable open-source frameworks implement static YaRN, which means the scaling factor remains constant regardless of input length
YaRN is currently supported by several inference frameworks, e.g., transformers, vllm, ktransformers and sglang.
All the notable open-source frameworks implement static YaRN, which means the scaling factor remains constant regardless of input length
Modifying the model configuration file
if the typical context length for your application is 524,288 tokens
It is also recommended to modify the factor as needed. For example, if the typical context length for your application is 524,288 tokens, it would be better to set factor as 2.0.
extensible to 1M via YaRN
For longer inputs, apply YaRN RoPE scaling via --hf-overrides and raise --max-model-len.
For longer inputs, apply YaRN RoPE scaling via --hf-overrides and raise --max-model-len.
Pick factor to match your real workload — 2.0 covers ~524K, 4.0 covers ~1M — since YaRN at higher factors degrades short-context quality.
apply YaRN RoPE scaling via --hf-overrides and raise --max-model-len
it applies the model card's YaRN RoPE configuration under text_config
For long-horizon tasks where the total length (including both input and output) exceeds this limit, we recommend using RoPE scaling techniques to handle long texts effectively, e.g., YaRN.
we recommend using RoPE scaling techniques to handle long texts effectively., e.g., YaRN.
we recommend using RoPE scaling techniques to handle long texts effectively., e.g., YaRN.
Filed alongside
Other methods under model architecture :: positional encoding.