Open Models Almanac
Methodsinference & servingKV cache management

implementation detail · filed under inference & serving

Language-model-only serving mode

Language-model-only serving mode runs a multimodal model without loading its vision encoder, serving text requests only.1 It frees the memory the encoder would occupy for KV cache, raising text throughput.2

Trade-off It trades multimodal capability for higher text throughput under high concurrency.3

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. use --language-model-only to skip loading the vision encoder — Qwen/Qwen3.5-397B-A17B — 397B / 17B active · MOE · 256K ctx
  2. and free up memory for KV cache — Qwen/Qwen3.5-397B-A17B — 397B / 17B active · MOE · 256K ctx
  3. For maximum text throughput under high concurrency, use --language-model-only — Qwen/Qwen3.5-397B-A17B — 397B / 17B active · MOE · 256K ctx

Also called language-model-only, Text-Only.

sources
3
models
2
lab adopt it
1
strongest
used

Documented in

Evidence

3 quotes from 3 sources, one card per model, each opening with its strongest.

How sources treat it
optional 2used 1

Qwen3.5-397B-A17B

Alibaba (Qwen)adopted2 sources

The following command skips the vision encoder and multimodal profiling to free up memory for additional KV cache

optionalinference servingin Qwen3.5-35B-A3BQwen Team
1 more from this model

use --language-model-only to skip loading the vision encoder and free up memory for KV cache, and enable Expert Parallelism.

usedinference servingin vLLMvLLM

Qwen3.6-35B-A3B

Alibaba (Qwen)adopted1 source

The following command skips the vision encoder and multimodal profiling to free up memory for additional KV cache: - vllm serve ... --language-model-only

optionalinference servingin Qwen3.6-35B-A3BQwen Team
↑