single mention · model architecture
Local Block
The block containing the current position is always selected during training and inference.
The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See fixed-pattern sparse attention for the methods filed beside it.
Filed under model architecture › token mixer › sparse attention › fixed-pattern sparse attention