Open Models Almanac
Methodsmodel architecturepositional encoding

methods · model architecture

positional encoding

24 methods filed at this node or below it, from the sources of 20 models.

In practice

3 features checked against each model’s own code and config.json, apart from its documents. A column’s page has the verdict and the lines.

adopteddiscussednot usedblank: documents silent✓in its code✕not in its code
ModelPartial RoPEYaRN RoPE scalingLayers without positional encoding (NoPE)
Mistral-Large-4in its codein its codein its code
DeepSeek-V4.1-Flash-1017in its codein its codecode unclear
DeepSeek-V4.1-Flashin its codein its codecode unclear
DeepSeek-V4-Flash-Vision-Expin its codein its codenot in its code
DeepSeek-V4-Pro-0813in its codein its codenot in its code
DeepSeek-V4-Flash-0731in its codein its codenot in its code
Laguna-S-2.1in its codein its codenot in its code
DeepSeek-V4-Flashin its codein its codenot in its code
DeepSeek-V4-Proin its codein its codenot in its code
DeepSeek-V3.2in its codein its codenot in its code
LFM3-1.7Bin its codenot in its codenot in its code
MiniMax-M3.1-Flashin its codecode unclearnot in its code
Step-5-Previewin its codenot in its codecode unclear
Kolibri-1not in its codecode unclearin its code
MiMo-V2.6-Flashin its codenot in its codenot in its code
MiMo-V2.6-Proin its codenot in its codenot in its code
Hy4-previewin its codenot in its codenot in its code
Qwen3.8-Flash-Nextin its codenot in its codenot in its code
GLM-5.3in its codenot in its codenot in its code
Kimi K3in its codecode unclearnot in its code
GLM-5.2in its codenot in its codenot in its code
MiniMax-M3in its codecode unclearnot in its code
Step-3.7-Flashin its codenot in its codenot in its code
MiMo-V2.5in its codenot in its codenot in its code
MiMo-V2.5-Proin its codenot in its codenot in its code
Qwen3.6-35B-A3Bin its codenot in its codenot in its code
Gemma 4 31Bin its codenot in its codenot in its code
Qwen3.5-397B-A17Bin its codenot in its codenot in its code
gpt-oss-120bnot in its codein its codenot in its code
NVIDIA-Nemotron-3.5-Super-VL-120B-A12Bcode unclearnot in its codecode unclear
GLM-5.3-Flashnot in its codenot in its codenot in its code
NVIDIA-Nemotron-3.5-Lightning-30B-A3Bcode unclearnot in its codecode unclear
Xiaomi-Robotics-1not in its codenot in its codenot in its code
Inklingnot in its codenot in its codecode unclear
Hy3not in its codenot in its codenot in its code
NVIDIA-Nemotron-3-Ultra-550B-A55Bnot in its codecode unclearcode unclear
MolmoAct 2not in its codenot in its codenot in its code
X-VLAnot in its codenot in its codenot in its code
OpenVLA / OpenVLA-OFTcode unclearnot in its codenot in its code

In this branch 24

Everything filed at this node or below it, with one collapsible heading per child node.

filed here 24

Rotary Position Embedding core · 6 sources · 6 quotes
No Position Encoding core · 3 sources · 4 quotes
Partial rotary position embeddings core · 3 sources · 3 quotes
Learned relative attention core · 2 sources · 2 quotes
LeRoPE core · 2 sources · 2 quotes
YaRN used · 11 sources · 11 quotes
Static YaRN used · 6 sources · 6 quotes
2D rotary position embeddings used · 2 sources · 2 quotes
Proportional Rotary Position Embedding used · 2 sources · 2 quotes
RoPE scaling optional · 2 sources · 2 quotes
YaRN scaling-factor selection optional · 2 sources · 3 quotes
13 from one source, or only mentioned as not used
Alternating RoPE/NoPE layers core · 1 source · 1 quote
Per-layer-type rotary position scales core · 1 source · 1 quote
UV positional encodings core · 1 source · 1 quote
2D coordinate-based positional embeddings used · 1 source · 1 quote
Axial 2D-RoPE used · 1 source · 1 quote
RoPE positional-index offset used · 1 source · 1 quote
RoPE scaling for context extension (YaRN) optional · 1 source · 1 quote
YaRN configuration optional · 1 source · 1 quote
Gated attention with partial RoPE evaluated · 1 source · 1 quote
Temporal-Modality Rotary Position Embedding not used · 1 source · 1 quote

By model

Which of this branch's methods each model's own documents describe, and how strongly. Under each model: its strongest treatment anywhere in the branch.

Modelmethods
DeepSeek-V4.1-Flash used2D rotary position embeddings used—
DeepSeek-V4-Flash usedRotary Position Embedding used—
DeepSeek-V4-Pro usedRotary Position Embedding used—
Gemma 4 31B used2D coordinate-based positional embeddings usedAxial 2D-RoPE usedProportional Rotary Position Embedding used—
Inkling coreLearned relative attention coreLearned input-dependent relative-position bias used—
Kimi K3 coreNo Position Encoding core—
Kolibri-1 coreNo Position Encoding coreStatic YaRN used—
LFM3-1.7B coreFull RoPE in sliding-window attention layers corePartial rotary position embeddings core—
Laguna-S-2.1 corePer-layer-type rotary position scales coreRotary Position Embedding usedGated attention with partial RoPE evaluated—
LingBot-VLA 2.0 coreShared spatial and modality-specific positional embeddings coreUV positional encodings core—
MiMo-V2.5 usedRotary Position Embedding used—
MiniMax-M3 usedRotary Position Embedding used—
Mistral-Large-4 coreAlternating RoPE/NoPE layers coreLeRoPE coreYaRN evaluated—
MolmoAct 2 coreRotary Position Embedding core—
NVIDIA-Nemotron-3.5-Lightning-30B-A3B coreNo Position Encoding core—
Qwen3.5-397B-A17B usedStatic YaRN usedRoPE scaling optionalRoPE scaling for context extension (YaRN) optionalYaRN optionalYaRN configuration optionalYaRN scaling-factor selection optionalTemporal-Modality Rotary Position Embedding not used—
Qwen3.6-35B-A3B optionalRoPE scaling optionalYaRN optionalYaRN scaling-factor selection optional—
Qwen3.8-Flash-Next usedPartial rotary position embeddings usedStatic YaRN usedYaRN optional—
Xiaomi-Robotics-1 usedRoPE positional-index offset used—
gpt-oss-120b usedRotary Position Embedding usedYaRN used—
↑