Token normalization for N-gram vocabulary compression

Token normalization used to compress the vocabulary for N-gram embeddings.

source: 1
model: 1
labs adopt it: 0
strongest: evaluated

How sources treat it

One count per evidence span, weakest treatment to strongest.

evaluated 1

Documented in

Qwen3.8-Flash-Next

Evidence

1 span quoted from the sources, strongest treatment first.

including but not limited to token normalization for vocabulary compression ... Despite these efforts, we observed no consistent performance gains in our training recipe.

evaluateddata curationin Qwen3.8-NextQwen

On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability·Qwen·2026-08-31

Filed alongside

Other methods under model architecture :: context capacity.

N-gram embedding host-memory offload and prefetch N-gram embedding lookup Engram 1M-token context window Progressive context extension 256K context window Multi-head hashing Row-wise distributed partitioning of embedding tables Single-layer N-gram embedding placement