specific method · filed under inference & serving
Attention data parallelism
Applies data parallelism to attention, with the evidence noting it avoids copying KV across ranks.
Also called Attention Data Parallelism (DP), Data Parallel attention, --enable-dp-attention, DP-attention.
- sources
- 3
- models
- 3
- labs adopt it
- 3
- strongest
- used
How sources treat it
One count per evidence span, weakest treatment to strongest.
Documented in
Further reading
Picked by hand, not extracted: where to read more, not evidence for anything on this page.
- DeepSeek V3 Support - SGLang Documentation docs docs.sglang.iodocuments --enable-dp-attention
Evidence
4 spans quoted from the sources, strongest treatment first.
DP-attention is primarily introduced to prevent copying KV across different ranks
We utilized a hybrid approach combining Attention Data Parallelism (DP) and MoE Expert Parallelism (EP)
enable_attention_dp: true
--enable-dp-attention
Filed alongside
Other methods under inference & serving :: serving parallelism.