Open Models Almanac
Methodsmodel architecturetoken mixersoftmax attentionglobal attention

implementation detail · filed under model architecture

Unified Keys and Values

An implementation detail in which global attention layers use unified keys and values to optimize memory for long contexts.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See global attention for the methods filed beside it.

Also called unified KV for global attention layers.

sources
2
model
1
lab adopt it
1
strongest
core

Documented in

Google DeepMind

Evidence

2 quotes from 2 sources, one card per model, each opening with its strongest.

How sources treat it
used 1core 1

Gemma 4 31B

Google DeepMindadopted2 sources

To optimize memory for long contexts, global layers feature unified Keys and Values, and apply Proportional RoPE (p-RoPE).

coremodel architecturein Gemma 4Google DeepMind
1 more from this model

To optimize memory for long contexts, global layers feature unified Keys and Values, and apply Proportional RoPE (p-RoPE).

usedmodel architecturein Gemma 4Google DeepMind
↑