single mention · model architecture
Two-stage sparse-attention training
A two-stage training strategy for introducing sparse attention, with the stages not specified in the evidence.
The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See sparse attention for the methods filed beside it.
Also called Two-stage introduction of sparse attention, two-stage training method.
Filed under model architecture › token mixer › sparse attention