single mention · model architecture
Dense Attention Fallback
Uses dense attention as a per-layer alternative when the usual attention pattern is not applied.
The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See hybrid layer stacking for the methods filed beside it.
Also called dense attention as a per-layer fallback.
Filed under model architecture › token mixer › hybrid layer stacking