single mention · not yet filed
The sources on this page only mention it in passing.
Methods the sources say this one serves — as a part, a setting, a variant or an enabler.
We place one such full-attention layer in every four layers, with the other three layers using GDN.