single mention · optimization
Intra-device context parallelism
An SM-level planner partitions a sequence across the SMs of one rank, computes segment transitions in parallel, and merges them to recover exact initial states.
The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See training parallelism for the methods filed beside it.
Filed under optimization › training parallelism