Open Models Almanac
Methods

single mention · not yet filed

Global attention with a dense feed-forward network in the first Transformer block

The sources on this page only mention it in passing.

Also called global attention with a dense FFN.