implementation detail · filed under inference & serving
Language-model-only serving mode
Language-model-only serving mode runs a multimodal model without loading its vision encoder, serving text requests only.1 It frees the memory the encoder would occupy for KV cache, raising text throughput.2
Trade-off It trades multimodal capability for higher text throughput under high concurrency.3
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
use --language-model-only to skip loading the vision encoder
— Qwen/Qwen3.5-397B-A17B — 397B / 17B active · MOE · 256K ctxand free up memory for KV cache
— Qwen/Qwen3.5-397B-A17B — 397B / 17B active · MOE · 256K ctxFor maximum text throughput under high concurrency, use --language-model-only
— Qwen/Qwen3.5-397B-A17B — 397B / 17B active · MOE · 256K ctx
Also called language-model-only, Text-Only.