Open Models Almanac
Methodsmodel architecturemultimodal architecture

single mention · model architecture

Decoupled Encoder Process (DEP)

Decoupled Encoder Process (DEP) splits ViT and text training into separate pipeline-parallel stages and balances vision forward and backward passes across them.1 It further decomposes ViT computation, running the first micro-batches' forward passes upfront and scheduling the rest into pipeline bubbles.2

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. splits ViT and text training into separate stages and balances vision forward and backward passes across PP stages — k3 tech report
  2. The ViT forward passes of the first PP micro-batches are executed synchronously upfront, the remaining forward passes are scheduled into pipeline bubbles — k3 tech report

Also called decoupled encoder process.

Filed under model architecture › multimodal architecture