Open Models Almanac
Methods

single mention · not yet filed

Tri-hybrid layer mix of gated short convolutions, sliding-window attention, and global attention

The sources on this page only mention it in passing.

Also called Tri-hybrid layer mix.