Techniquesmodel architecturecontext capacity

specific method · filed under model architecture

Multi-head hashing

A hashing method used to perform N-gram embedding lookup.

source: 1
model: 1
lab adopt it: 1
strongest: used

How sources treat it

One count per evidence span, weakest treatment to strongest.

used 1

Documented in

Qwen3.8-Flash-Next

Evidence

1 span quoted from the sources, strongest treatment first.

We use multi-head hashing for embedding lookup

usedmodel architecturein Qwen3.8-NextQwen

On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability·Qwen·2026-08-31

Filed alongside

Other methods under model architecture :: context capacity.

N-gram embedding host-memory offload and prefetch N-gram embedding lookup Engram 1M-token context window Progressive context extension 256K context window Row-wise distributed partitioning of embedding tables Single-layer N-gram embedding placement Token normalization for N-gram vocabulary compression