single mention · not yet filed
Two-stage contextual-parallel communication for compressed attention
A two-stage communication approach for context-parallel compressed attention, designed because compression needs consecutive KV entries that can straddle rank boundaries and produce varying compressed lengths.1 Each rank first sends its trailing uncompressed KV entries to its neighbor for compression, then an all-gather collects the locally compressed KV entries, which a select-and-pad operator reorganizes into the full compressed set.2
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
the compression requires consecutive KV entries, which may straddle the boundary between two neighboring CP ranks
— DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligenceeach rank sends its last uncompressed KV entries to rank . Then, rank compresses some of these received entries together with its local uncompressed KV entries
— DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
Also called two-stage communication approach.