single mention · not yet filed
The sources on this page only mention it in passing.
Methods the sources say this one serves — as a part, a setting, a variant or an enabler.
Fusing the residual read and the final normalization before the LM head into one gated read operation further reduces the gradient norm