Taxonomymodel architecturepositional encoding

taxonomy node · level 2

positional encoding

14 methods filed at this node or below it, from the sources of 14 models.

model architecture :: positional encoding

Matching aids for the classifier: RoPE; rotary position embedding; NoPE; YaRN.

In this branch 14

Everything filed at this node or below it, with one collapsible heading per child node.

filed here 14

Relative attention core · 2 sources · 2 quotes

No Position Encoding core · 1 source · 2 quotes

Omitting RoPE in attention layers core · 1 source · 1 quote

Per-layer-type rotary position scales core · 1 source · 1 quote

YaRN used · 12 sources · 19 quotes

Rotary Position Embedding used · 5 sources · 5 quotes

2D rotary position embedding used · 3 sources · 3 quotes

Proportional Rotary Position Embedding used · 2 sources · 2 quotes

2D coordinate-based positional embeddings used · 1 source · 1 quote

Learned input-dependent relative position bias used · 1 source · 2 quotes

Partial RoPE used · 1 source · 1 quote

RoPE scaling optional · 3 sources · 3 quotes

Gated attention with partial RoPE evaluated · 1 source · 1 quote

Temporal-Modality Rotary Position Embedding not used · 1 source · 1 quote

By model

Which of this branch's techniques each model's own documents describe, and how strongly. Under each model: its strongest treatment anywhere in the branch.

Model	techniques
DeepSeek-V4.1-Flash used	2D rotary position embedding used—
DeepSeek-V4-Flash used	Rotary Position Embedding used—
MiMo-V2.5 used	Rotary Position Embedding used—
MiniMax-M3 used	Rotary Position Embedding used—
DeepSeek-V4-Pro used	Rotary Position Embedding used—
Gemma 4 31B used	2D coordinate-based positional embeddings used2D rotary position embedding usedProportional Rotary Position Embedding used—
Inkling core	Relative attention coreLearned input-dependent relative position bias used—
Kimi K3 core	No Position Encoding core—
Laguna-S-2.1 core	Per-layer-type rotary position scales coreRotary Position Embedding usedGated attention with partial RoPE evaluated—
NVIDIA-Nemotron-3.5-Lightning-30B-A3B core	Omitting RoPE in attention layers core—
Qwen3.5-397B-A17B used	YaRN usedRoPE scaling optionalTemporal-Modality Rotary Position Embedding not used—
Qwen3.6-35B-A3B optional	RoPE scaling optionalYaRN optional—
Qwen3.8-Flash-Next used	Partial RoPE usedYaRN used—
gpt-oss-120b used	Rotary Position Embedding usedYaRN used—