single mention · model architecture
Decoupled Encoder Process (DEP)
Decoupled Encoder Process (DEP) splits ViT and text training into separate pipeline-parallel stages and balances vision forward and backward passes across them.1 It further decomposes ViT computation, running the first micro-batches' forward passes upfront and scheduling the rest into pipeline bubbles.2
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
splits ViT and text training into separate stages and balances vision forward and backward passes across PP stages
— k3 tech reportThe ViT forward passes of the first PP micro-batches are executed synchronously upfront, the remaining forward passes are scheduled into pipeline bubbles
— k3 tech report
Also called decoupled encoder process.
Filed under model architecture › multimodal architecture