Open Models Almanac
Methodsmodel architecture

specific method · filed under model architecture

Causal Encoder-Decoder (CED)

A Causal Encoder-Decoder architecture treats the bottom half of the Transformer's layers as a causal encoder and the upper half as a decoder.1 For global attention, the decoder's KV entries are projected directly from the encoder's final hidden state, so prefill only computes the first half of the layers.2,3

Trade-off It cuts prefill computation by nearly half while keeping performance comparable to the baseline.4

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. CED treats the bottom 𝐿/2 layers of the Transformer as the causal encoder — DeepSeek V41 Tech Report
  2. they are projected directly from the hidden state of the(𝐿/2)-th layer — DeepSeek V41 Tech Report
  3. This design allows CED to compute only the first half of the layers during the prefill phase — DeepSeek V41 Tech Report
  4. reduces nearly half of the prefill computation while maintaining performance comparable to the baseline — DeepSeek V41 Tech Report

Also called Causal Encoder-Decoder architecture, Causal Encoder-Decoder (CED) architecture.

sources
5
models
2
lab adopt it
1
strongest
core

Documented in

Evidence

6 quotes from 5 sources, one card per model, each opening with its strongest.

How sources treat it
core 6

DeepSeek-V4.1-Flash

DeepSeekadopted4 sources

DeepSeek-V4.1-Flash adopts a Causal Encoder-Decoder (CED) architecture: a 40-layer Transformer organized as a 20-layer causal encoder followed by a 20-layer decoder.

coremodel architecturein DeepSeek-V4.1-FlashDeepSeek
4 more from this model

the KV entries are not derived from their respective hidden states ... Instead, they are projected directly from the hidden state of the (L/2)-th layer

coreunclearin DeepSeek-V4.1-FlashDeepSeek

we adopt a Causal Encoder-Decoder (CED) architecture, in which decoder global KV is projected from the final encoder hidden states

coreunclearin DeepSeek-V4.1-FlashDeepSeek

New Causal Encoder–Decoder architecture: just 8B active parameters for input, 16B for output.

coremodel architecturein DeepSeek-V4.1-FlashDeepSeek

the first built on the company's Causal Encoder-Decoder (CED) architecture

coremodel architecturein DeepSeek-V4.1-FlashDeepSeek

Further reading

Picked by hand, not extracted: where to read more, not evidence for anything on this page.

↑