Open Models Almanac
Methodsmodel architecturetoken mixersoftmax attention

specific method · filed under model architecture

Blockwise causal attention

Applies causal attention across sequence blocks, with temporal structure specified in one mention.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See softmax attention for the methods filed beside it.

sources
2
model
1
lab adopt it
1
strongest
core

Documented in

Robbyant (Ant Group)

Evidence

2 quotes from 2 sources, one card per model, each opening with its strongest.

How sources treat it
used 1core 1
↑