Open Models Almanac
Methodsmodel architecturetoken mixersparse attentionfixed-pattern sparse attention

single mention · model architecture

Local Block

The block containing the current position is always selected during training and inference.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See fixed-pattern sparse attention for the methods filed beside it.

Filed under model architecture › token mixer › sparse attention › fixed-pattern sparse attention