Open Models Almanac
Methodsmodel architecturecontext capacity

implementation detail · filed under model architecture

Asynchronous prefetching of offloaded N-gram embeddings

Asynchronous prefetching overlaps the fetching of N-gram embeddings offloaded to host memory with the model's computation.1 It hides the latency of host-memory lookups so that using the offloaded embedding table adds little overhead.2

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. the embedding table can be offloaded to host memory and overlapped with model computation through asynchronous prefetching — Qwen3.8-Flash-Next
  2. looks up a table using the local context to scale model capacity with very little extra computation — Qwen3.8-Flash-Next

Also called asynchronous prefetching.

sources
2
model
1
lab adopt it
1
strongest
used

Documented in

Alibaba (Qwen)

Evidence

2 quotes from 2 sources, one card per model, each opening with its strongest.

How sources treat it
optional 1used 1
↑