Open Models Almanac
Methodsmodel architecturecontext capacity

single mention · model architecture

Host-memory-prefetched N-gram embedding tables

A single N-gram embedding layer has tables prefetched from host memory to add capacity outside the backbone.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See context capacity for the methods filed beside it.

Filed under model architecture › context capacity