Open Models Almanac
Methodsoptimizationoptimizer

single mention · optimization

Asynchronous Micro-Group pipeline

An asynchronous Micro-Group pipeline reconstructs each Muon-owned matrix via fused All-to-All across TP ranks.1 It lets the optimizer run on the full matrix even though no single tensor-parallel rank owns it, keeping the step mathematically equivalent to single-device Muon.2

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. an asynchronous Micro-Group pipeline reconstructs each Muon-owned matrix via fused All-to-All across TP ranks — On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability
  2. Each owner then runs a step mathematically equivalent to single-device Muon — On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability

Also called an asynchronous Micro-Group pipeline.

Filed under optimization › optimizer