Open Models Almanac
Methodsoptimizationtraining parallelism

single mention · optimization

Cache-based pipeline communication

Cache-based pipeline communication transfers tensors between pipeline-parallel stages by sending only newly produced blocks incrementally instead of full activations.1 The transferred blocks are freed as soon as their micro-batch completes, minimizing resident memory.2

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. only newly generated blocks are incrementally transferred between stages — k3 tech report
  2. released as soon as the micro-batch finishes, reaching the theoretical lower bound on memory footprint — k3 tech report

Filed under optimization › training parallelism