implementation detail · filed under model architecture
Unified Keys and Values
A memory-optimization implementation detail in global-attention layers that unifies keys and values.
Also called unified KV for global attention layers.
- sources
- 2
- model
- 1
- lab adopt it
- 1
- strongest
- core
How sources treat it
One count per evidence span, weakest treatment to strongest.
used 1core 1
Documented in
Evidence
2 spans quoted from the sources, strongest treatment first.
To optimize memory for long contexts, global layers feature unified Keys and Values, and apply Proportional RoPE (p-RoPE).
coremodel architecturein Gemma 4Google DeepMind
To optimize memory for long contexts, global layers feature unified Keys and Values, and apply Proportional RoPE (p-RoPE).
usedmodel architecturein Gemma 4Google DeepMind
Filed alongside
Other methods under model architecture :: token mixer :: softmax attention :: global attention.