specific method · filed under model architecture
Proportional Rotary Position Embedding
A proportional variant of rotary position embedding used in global layers in the cited configuration.
Also called Proportional RoPE (p-RoPE), proportional rotary position embeddings.
- sources
- 2
- model
- 1
- lab adopt it
- 1
- strongest
- used
How sources treat it
One count per evidence span, weakest treatment to strongest.
used 2
Documented in
Evidence
2 spans quoted from the sources, strongest treatment first.
global layers feature unified Keys and Values, and apply Proportional RoPE (p-RoPE).
usedmodel architecturein Gemma 4Google DeepMind
To optimize memory for long contexts, global layers feature unified Keys and Values, and apply Proportional RoPE (p-RoPE).
usedmodel architecturein Gemma 4Google DeepMind
Filed alongside
Other methods under model architecture :: positional encoding.
YaRNRotary Position Embedding2D rotary position embeddingRoPE scalingLearned input-dependent relative position biasNo Position EncodingRelative attention2D coordinate-based positional embeddingsGated attention with partial RoPEOmitting RoPE in attention layersPartial RoPEPer-layer-type rotary position scalesTemporal-Modality Rotary Position Embedding