Open Models Almanac
Methodsmodel architecturepositional encoding

specific method · filed under model architecture

YaRN

YaRN is a positional-encoding method that extends a model's context length by rescaling the RoPE frequencies.1

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. used YaRN to rescale the RoPE frequencies — technical report

Also called YaRN context length extension, YaRN context window extension, YaRN RoPE scaling, YaRN RoPE scaling for long context, YaRN rotary-position-frequency rescaling, YaRN (Yet another RoPE extensioN).

sources
11
models
5
labs adopt it
2
strongest
used

In practice

Checked as “YaRN RoPE scaling” against each model’s own code and config.json, apart from its documents.

✓in its code✕not in its code
ModelCodeIts documentsParametersLink
Mistral-Large-4in its codediscussed
rope_parameters={"attention_factor": 1.0, "beta_fas…

rope_parameters.rope_type is 'yarn' and the rotary embedding module dispatches to the YaRN init function via ROPE_INIT_FUNCTIONS, with mscale applied to attention scaling.

text_config.rope_parameters = {"attention_factor": 1.0, "beta_fast": 32, "beta_slow": 1, "factor": 2048, "o…
self.rope_type = self.config.rope_parameters["rope_type"]
        rope_init_fn: Callable = self.compute_default_rope_parameters
        if self.rope_type != "default":
            rope_init_fn = ROPE_INIT_FUNCTIONS[self.rope_type]
modeling_mistral_large4.py · L84–87
DeepSeek-V4.1-Flash-1017in its codenot stated
rope_scaling={"rope_type": "yarn", "factor": 16,…

rope_scaling in text_config sets rope_type to yarn with factor 16 and related YaRN parameters, applied via the attention module's rotary_emb (cos_sin_cache) referenced throughout the code.

text_config.rope_scaling = {"rope_type": "yarn", "factor": 16, "beta_fast": 32, "beta_slow": 1, "origina…
attn.rotary_emb.cos_sin_cache,
dspark.py · L284
DeepSeek-V4.1-Flashin its codenot stated
rope_scaling={"rope_type": "yarn", "factor": 16,…

rope_scaling in text_config sets rope_type to yarn with factor 16 and associated beta parameters, and the model's rotary embedding (rotary_emb.cos_sin_cache) consumes this config.

text_config.rope_scaling = {"rope_type": "yarn", "factor": 16, "beta_fast": 32, "beta_slow": 1, "origina…
attn.rotary_emb.cos_sin_cache,
dspark.py · L284
DeepSeek-V4-Flash-Vision-Expin its codenot stated
rope_scaling={"beta_fast": 32, "beta_slow": 1, "…

rope_scaling type is yarn and is applied to the compress rope-type branch (CSA/HCA layers) via ROPE_INIT_FUNCTIONS dispatch keyed by rope_type.

rope_scaling = {"beta_fast": 32, "beta_slow": 1, "factor": 16, "original_max_position_embedd…
compress.setdefault("rope_type", "default")
            if compress["rope_type"] == "yarn":
                compress.setdefault("attention_factor", 1.0)
configuration_deepseek_v4.py · L317–319
rope_init_fn = self.compute_default_rope_parameters
            if self.rope_type[layer_type] != "default":
                rope_init_fn = ROPE_INIT_FUNCTIONS[self.rope_type[layer_type]]
modeling_deepseek_v4.py · L104–106
DeepSeek-V4-Pro-0813in its codenot stated
rope_scaling={"beta_fast": 32, "beta_slow": 1, "…

rope_scaling type is yarn and is applied to the compress-branch rope used by CSA/HCA attention layers via the ROPE_INIT_FUNCTIONS dispatch keyed by rope_type

rope_scaling = {"beta_fast": 32, "beta_slow": 1, "factor": 16, "original_max_position_embedd…
compress.setdefault("rope_type", "default")
            if compress["rope_type"] == "yarn":
                compress.setdefault("attention_factor", 1.0)
configuration_deepseek_v4.py · L317–319
rope_init_fn = self.compute_default_rope_parameters
            if self.rope_type[layer_type] != "default":
                rope_init_fn = ROPE_INIT_FUNCTIONS[self.rope_type[layer_type]]
modeling_deepseek_v4.py · L104–106
DeepSeek-V4-Flash-0731in its codenot stated
rope_scaling={"beta_fast": 32, "beta_slow": 1, "…

rope_scaling specifies type yarn with factor 16 and this config's compress-branch RoPE parameters set rope_type yarn, which the rotary embedding dispatches to the YaRN init function.

rope_scaling = {"beta_fast": 32, "beta_slow": 1, "factor": 16, "original_max_position_embedd…
rope_init_fn = self.compute_default_rope_parameters
            if self.rope_type[layer_type] != "default":
                rope_init_fn = ROPE_INIT_FUNCTIONS[self.rope_type[layer_type]]
modeling_deepseek_v4.py · L104–106
Laguna-S-2.1in its codenot stated
rope_type=yarn · factor=128.0

The full_attention rope_parameters entry sets rope_type yarn with factor/beta_fast/beta_slow, and the rotary embedding dispatches to ROPE_INIT_FUNCTIONS for non-default types.

rope_parameters.full_attention.rope_type = "yarn"rope_parameters.full_attention.factor = 128.0
self.rope_type[layer_type] = rope_params["rope_type"]
            rope_init_fn: Callable = self.compute_default_rope_parameters
            if self.rope_type[layer_type] != "default":
                rope_init_fn = ROPE_INIT_FUNCTIONS[self.rope_type[layer_type]]
modeling_laguna.py · L79–82
DeepSeek-V4-Flashin its codenot stated
rope_scaling={"beta_fast": 32, "beta_slow": 1, "…

rope_scaling type is yarn in the config and compress-branch rope_parameters are built with this yarn configuration (factor 16, beta_fast/slow) consumed by ROPE_INIT_FUNCTIONS for the 'compress' layer type.

rope_scaling = {"beta_fast": 32, "beta_slow": 1, "factor": 16, "original_max_position_embedd…
if compress["rope_type"] == "yarn":
                compress.setdefault("attention_factor", 1.0)
            self.rope_parameters = {"main": main, "compress": compress}
configuration_deepseek_v4.py · L318–320
DeepSeek-V4-Proin its codenot stated
rope_scaling={"beta_fast": 32, "beta_slow": 1, "…

rope_scaling.type is yarn and is wired into the compress rope-type parameters that the YaRN-capable ROPE_INIT_FUNCTIONS dispatch consumes for compressed-attention layers.

rope_scaling = {"beta_fast": 32, "beta_slow": 1, "factor": 16, "original_max_position_embedd…
rope_init_fn = self.compute_default_rope_parameters
            if self.rope_type[layer_type] != "default":
                rope_init_fn = ROPE_INIT_FUNCTIONS[self.rope_type[layer_type]]
modeling_deepseek_v4.py · L104–106
DeepSeek-V3.2in its codenot stated
type=yarn · factor=40

rope_scaling.type is 'yarn' and the rotary embedding init dispatches to the YaRN ROPE_INIT_FUNCTIONS, with mscale applied to the attention scaling.

rope_scaling.type = "yarn"rope_scaling.factor = 40
self.rope_type = self.config.rope_parameters["rope_type"]
        rope_init_fn: Callable = self.compute_default_rope_parameters
        if self.rope_type != "default":
            rope_init_fn = ROPE_INIT_FUNCTIONS[self.rope_type]
modeling_deepseek_v32.py · L75–78
gpt-oss-120bin its codeadopted
rope_type=yarn · factor=32.0

rope_scaling specifies rope_type yarn with factor 32.0, and the rotary embedding module dispatches to the YaRN init function

rope_scaling.rope_type = "yarn"rope_scaling.factor = 32.0
self.rope_type = self.config.rope_parameters["rope_type"]
        rope_init_fn: Callable = self.compute_default_rope_parameters
        if self.rope_type != "default":
            rope_init_fn = ROPE_INIT_FUNCTIONS[self.rope_type]
modeling_gpt_oss.py · L156–159
LFM3-1.7Bnot in its codenot stated
rope_parameters.full_attention.rope_type=default · rope_parameters.sliding_attention.rope_type=default

rope_parameters entries specify rope_type "default" for both layer types, not YaRN, and no YaRN scaling code path is exercised.

rope_parameters.full_attention.rope_type = "default"rope_parameters.sliding_attention.rope_type = "default"
MiniMax-M3.1-Flashcode unclearnot stated
verdict and lines

rope_parameters is not populated with an explicit rope_scaling/type in the given config (only rope_theta and rotary_dim appear at top level of text_config), so whether YaRN is switched on cannot be determined from what is given.

NVIDIA-Nemotron-3.5-Super-VL-120B-A12Bnot in its codenot stated
verdict and lines

No rope_scaling or rope_parameters key is present anywhere in the config, so no YaRN scaling is configured.

Step-5-Previewnot in its codenot stated
rope_scaling={"rope_type": "llama3", "factor": 1…

rope_scaling.rope_type is 'llama3', not YaRN, so no YaRN scaling is applied despite yarn_only_types gating logic in code.

text_config.rope_scaling = {"rope_type": "llama3", "factor": 1.0, "original_max_position_embeddings": 10…
Kolibri-1code unclearnot stated
rope_theta=10000.0

rope_parameters is read from config but its contents (rope_type, possible yarn scaling) are not shown in the given config.json, which only lists rope_theta at top level.

rope_theta = 10000.0
        self.rope_type = self.config.rope_parameters["rope_type"]
        rope_init_fn: Callable = self.compute_default_rope_parameters
        if self.rope_type != "default":
            rope_init_fn = ROPE_INIT_FUNCTIONS[self.rope_type]
modeling_kolibri1.py · L403–406
MiMo-V2.6-Flashnot in its codenot stated
rope_type=default · type=default

rope_parameters/rope_type is set to "default" in this config, not YaRN, so no YaRN scaling is applied.

rope_parameters.rope_type = "default"rope_parameters.type = "default"
MiMo-V2.6-Pronot in its codenot stated
rope_type=default · type=default

rope_parameters/rope_type is set to "default", not YaRN, so no YaRN scaling is applied.

rope_parameters.rope_type = "default"rope_parameters.type = "default"
Hy4-previewnot in its codenot stated
rope_parameters={"rope_theta": 10000000, "rope_type…

rope_parameters.rope_type is set to 'default' rather than 'yarn', so although yarn-related mscale helper code exists it is not activated by this config.

rope_parameters = {"rope_theta": 10000000, "rope_type": "default"}
GLM-5.3-Flashnot in its codenot stated
verdict and lines

No rope_scaling or rope_parameters key of type YaRN appears in text_config, and the text attention path uses NoPE (no rotary applied at all), so YaRN scaling cannot apply.

Qwen3.8-Flash-Nextnot in its codeadopted
rope_type=default

rope_parameters.rope_type is set to "default", not YaRN, so no YaRN scaling is applied.

text_config.rope_parameters.rope_type = "default"
GLM-5.3not in its codenot stated
rope_parameters={"rope_theta": 8000000, "rope_type"…

rope_parameters.rope_type is "default", not yarn, so although yarn_apply_mscale code exists it is a no-op for this config.

rope_parameters = {"rope_theta": 8000000, "rope_type": "default"}
def yarn_apply_mscale(rope_parameters, scaling):
    if rope_parameters.get("rope_type", "default") != "default":
modeling_glm_moe_dsa.py · L288–289
NVIDIA-Nemotron-3.5-Lightning-30B-A3Bnot in its codenot stated
verdict and lines

No rope_scaling key is present in this config, so no YaRN scaling is configured.

Xiaomi-Robotics-1not in its codenot stated
rope_scaling={"mrope_interleaved": true, "mrope_…

rope_scaling in text_config sets rope_type to 'default' with mrope_section/mrope_interleaved for multimodal RoPE, not YaRN, so YaRN scaling is not enabled.

vlm_config.text_config.rope_scaling = {"mrope_interleaved": true, "mrope_section": [24, 20, 20], "rope_type": "defa…
Kimi K3code unclearnot stated
verdict and lines

The config exposes a rope_scaling attribute in KimiLinearConfig defaulting to None and it is not present as a key in the given text_config, and no rotary_emb / RoPE application code is shown (self.rotary_emb is set to None and never used), so YaRN scaling cannot be confirmed as active.

self.rotary_emb = None
modeling_kimi_linear.py · L403
Inklingnot in its codenot stated
verdict and lines

There is no rope_scaling key in the config and no RoPE scaling code of any kind; positions are encoded via a learned relative-bias projection, not rotary embeddings.

Hy3not in its codenot stated
rope_parameters={"rope_theta": 11158840.0, "rope_ty…

rope_parameters.rope_type is set to 'default', not yarn, so no YaRN scaling is applied.

rope_parameters = {"rope_theta": 11158840.0, "rope_type": "default"}
GLM-5.2not in its codenot stated
rope_parameters={"rope_theta": 8000000, "rope_type"…

rope_parameters.rope_type is "default" (not yarn), so although yarn_apply_mscale code exists, it is a no-op for this config.

rope_parameters = {"rope_theta": 8000000, "rope_type": "default"}
def yarn_apply_mscale(rope_parameters, scaling):
    if rope_parameters.get("rope_type", "default") != "default":
modeling_glm_moe_dsa.py · L288–289
NVIDIA-Nemotron-3-Ultra-550B-A55Bcode unclearnot stated
verdict and lines

No rope_scaling or rope_parameters key is present in the config, and the attention module doesn't reference rotary embeddings or scaling at all, so YaRN cannot be confirmed or ruled out from the given files.

MiniMax-M3code unclearnot stated
verdict and lines

rope_parameters is not given a literal value in the provided config.json (it is None in the class default) so whether YaRN scaling is active cannot be determined from these files.

self.rope_type = self.config.rope_parameters["rope_type"]
        rope_init_fn: Callable = self.compute_default_rope_parameters
        if self.rope_type != "default":
            rope_init_fn = ROPE_INIT_FUNCTIONS[self.rope_type]
modeling_minimax_m3_vl.py · L273–276
Step-3.7-Flashnot in its codenot stated
rope_scaling={"rope_type": "llama3", "factor": 2…

rope_scaling specifies rope_type 'llama3' (not YaRN) and the config has no YaRN rope_type entry; the 'yarn_only_types' key only names which layer types the llama3-style scaling applies to.

text_config.rope_scaling = {"rope_type": "llama3", "factor": 2.0, "original_max_position_embeddings": 13…
MolmoAct 2not in its codenot stated
rope_parameters={"rope_theta": 5000000.0, "rope_typ…

rope_parameters.rope_type is 'default' (no YaRN scaling type set), so the RoPE scaling code path for YaRN is not switched on.

text_config.rope_parameters = {"rope_theta": 5000000.0, "rope_type": "default"}
MiMo-V2.5not in its codenot stated
rope_scaling={"rope_type": "default", "type": "d…

rope_scaling type is 'default', not YaRN, so no YaRN scaling is applied.

rope_scaling = {"rope_type": "default", "type": "default"}
MiMo-V2.5-Pronot in its codenot stated
verdict and lines

rope_scaling is not set in this config (no key present) so no YaRN scaling is applied; rotary embedding falls back to the default type.

self.rope_type = "default"
modeling_mimo_v2.py · L448
Qwen3.6-35B-A3Bnot in its codeadopted
rope_type=default

rope_parameters.rope_type is "default", not yarn, so no YaRN scaling is applied.

text_config.rope_parameters.rope_type = "default"
Gemma 4 31Bnot in its codenot stated
rope_parameters={"full_attention": {"partial_rotary…

rope_parameters only declare rope_type 'proportional' and 'default', not YaRN, so no YaRN scaling is configured or applied.

text_config.rope_parameters = {"full_attention": {"partial_rotary_factor": 0.25, "rope_theta": 1000000.0, "…
Qwen3.5-397B-A17Bnot in its codeadopted
rope_parameters={"mrope_interleaved": true, "mrope_…

rope_parameters.rope_type is "default", not "yarn", so no YaRN scaling is switched on.

text_config.rope_parameters = {"mrope_interleaved": true, "mrope_section": [11, 11, 10], "rope_type": "defa…
X-VLAnot in its codenot stated
verdict and lines

There is no rope_scaling key in the config and no RoPE-scaling code (YaRN or otherwise) anywhere in the provided source, consistent with the model using learned absolute positions instead of RoPE.

OpenVLA / OpenVLA-OFTnot in its codenot stated
model_type=llama

There is no rope_scaling or rope_parameters key in text_config, so no YaRN scaling is configured.

text_config.model_type = "llama"

Documented in

Evidence

11 quotes from 11 sources, one card per model, each opening with its strongest.

How sources treat it
evaluated 1optional 9used 1

Qwen3.6-35B-A3B

Alibaba (Qwen)adopted5 sources

For longer inputs, apply YaRN RoPE scaling via --hf-overrides and raise --max-model-len.

optionalinference servingin Qwen3.6-27BQwen Team
4 more from this model

apply YaRN RoPE scaling via --hf-overrides and raise --max-model-len

optionalunclearin Qwen3.6-35B-A3BQwen Team

we recommend using RoPE scaling techniques to handle long texts effectively., e.g., YaRN.

optionalinference servingin Qwen3.6-27BQwen Team

we recommend using RoPE scaling techniques to handle long texts effectively., e.g., YaRN.

optionalinference servingin Qwen3.6-35B-A3BQwen Team

extensible to 1M via YaRN

optionalinference servingin Qwen3.6-35B-A3BAlibaba Cloud

Qwen3.5-397B-A17B

Alibaba (Qwen)adopted3 sources

For long-horizon tasks where the total length (including both input and output) exceeds this limit, we recommend using RoPE scaling techniques to handle long texts effectively, e.g., YaRN.

optionalinference servingin Qwen3.5-122B-A10BQwen Team
2 more from this model

YaRN is currently supported by several inference frameworks, e.g., transformers, vllm, ktransformers and sglang.

optionalinference servingin Qwen3.5-397B-A17BQwen Team

For longer inputs, apply YaRN RoPE scaling via --hf-overrides and raise --max-model-len.

optionalinference servingin Qwen3.5-397B-A17BQwen Team

Qwen3.8-Flash-Next

Alibaba (Qwen)adopted1 source

For long-horizon tasks where the total length (including both input and output) exceeds this limit, we recommend using RoPE scaling techniques to handle long texts effectively, e.g., YaRN.

optionalunclearin Qwen3.8-Flash-NextQwen Team

Mistral-Large-4

Mistral AIdiscussed1 source

we removed the attention windows and used YaRN to rescale the RoPE frequencies by 8×

evaluatedmodel architecturein Mistral Large 4Mistral AI

Further reading

Picked by hand, not extracted: where to read more, not evidence for anything on this page.

↑