specific method · filed under model architecture
Per-Layer Embeddings (PLE)
An approach that gives each decoder layer its own small embedding for every token to improve parameter efficiency.
Also called per-layer embeddings.
- sources
- 4
- model
- 1
- lab adopt it
- 1
- strongest
- core
How sources treat it
One count per evidence span, weakest treatment to strongest.
Documented in
Further reading
Picked by hand, not extracted: where to read more, not evidence for anything on this page.
- Per-Layer Embeddings (PLE) explainer sebastianraschka.com
Evidence
4 spans quoted from the sources, strongest treatment first.
PLE gives each decoder layer its own small embedding for every token.
The smaller models incorporate Per-Layer Embeddings (PLE) to maximize parameter efficiency in on-device deployments. Rather than adding more layers or parameters to the model, PLE gives each decoder layer its own small embedding for every token.
Rather than adding more layers or parameters to the model, PLE gives each decoder layer its own small embedding for every token.
E2B and E4B use per-layer embeddings as in Gemma 3n [Gemma Team, 2025b], making them 2.3B and 4.5B effective out of 5B and 8B total parameters respectively.
Filed alongside
Other methods under model architecture.