Qwen3.6-35B-A3B
Alibaba (Qwen)Pick factor to match your real workload — 2.0 covers ~524K, 4.0 covers ~1M — since YaRN at higher factors degrades short-context quality.
implementation detail · filed under model architecture
This implementation detail selects or adjusts YaRN's scaling factor based on target or typical context length and workload.
The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See positional encoding for the methods filed beside it.
Also called modify the factor as needed, Pick factor to match your real workload.
3 quotes from 2 sources, one card per model, each opening with its strongest.
Pick factor to match your real workload — 2.0 covers ~524K, 4.0 covers ~1M — since YaRN at higher factors degrades short-context quality.
It is also recommended to modify the factor as needed. For example, if the typical context length for your application is 524,288 tokens, it would be better to set factor as 2.0.
if the typical context length for your application is 524,288 tokens