| DeepSeek-V4.1-Flash-1017 used | — | — | — | Compressed KV caching used— | — | — | Provider failover routing used— | — | — | API request conversion, prompt encoding, and response parsing used— |
| GLM-5.3-Flash core | — | Multi-Token Prediction used— | Always-on thinking mode coreChat-template reasoning-effort resolution and injection defaultThinking mode defaultclear_thinking chat-template flag optional— | — | FP8 defaultMXFP8 block-scale regrouping at load usedFP8 KV-cache quantization optional— | KDA state and KV-cache layout pinning coreDisaggregated serving across encoding, prefill, and decode usedPrefill-decode disaggregation usedRound-robin routing for prefill-decode disaggregation used— | — | ROCm AITER sparse MLA attention backend used— | — | Video understanding and editing optionalVision-driven UI coding optional— |
| DeepSeek-V4.1-Flash core | — | Confidence-scheduled verification coreDSpark coreSemi-autoregressive draft generation core— | Capped linear reasoning-token length deduction coreContinuously controllable reasoning effort coreEffort-dependent exponential token-penalty schedule corePreset effort tiers mapped to scalar values defaultSystem-prompt reasoning-effort conditioning used— | Compressed KV caching coreSWA Bounded Replay core— | FP4 global KV-cache quantization core— | — | Independent replica placement using predicted resource availability coreLatency-sensitive execution class coreNode-local placement validation with hard admission control coreSCHED_IDLE prioritization for non-latency-sensitive tasks coreSub-NUMA partitioning with NUMA-local worker VMs core— | Mega-mHC used— | — | Agent Team mode core— |
| Hy4-preview default | Communication optimization used— | Multi-Token Prediction usedNEXTN speculative decoding usedSpeculative decoding used— | Chain-of-thought reasoning defaultThinking mode used— | — | FP8 usedLarge-model quantization and speculative sampling optional— | Tensor parallelism used— | — | Kernel fusion used— | — | Automatic tool choice usedhy_v4 tool-call parser used— |
| Step-5-Preview core | Offline generation optional— | Multi-layer EAGLE usedSpeculative decoding usedEAGLE optionalMulti-Token Prediction optional— | — | Prefix caching optionalHiCache mentioned— | FP8 optional— | Attention data parallelism mentioned— | — | — | — | Iterative tool execution with verification and revision core— |
| MiniMax-M3.1-Flash used | — | DSpark usedDSpark draft head optionalTemperature and nucleus sampling configuration optional— | Thinking mode optional— | — | — | — | Provider routing usedLoad balancing across providers mentioned— | — | — | — |
| GLM-5.3 default | — | Multi-Token Prediction used— | clear_thinking chat-template flag defaultThinking mode default— | Disabling prefix caching for benchmarking used— | FP8 defaultFP8 KV-cache quantization usedNVFP4 quantization of MoE expert linears optional— | — | Maximum-sequence tuning used— | — | Context management method used— | — |
| DeepSeek-V4-Flash-0731 used | — | DSpark usedChat Prefix Completion optionalSpeculative decoding mentioned— | Thinking mode used— | — | — | — | — | — | — | Function calling optionalJSON mode optional— |
| Hy3 core | — | Speculative decoding coreEAGLE optionalSpeculative sampling optional— | Thinking mode defaultReasoning parser usedChain-of-thought reasoning optional— | — | FP8 usedFP8 KV-cache quantization optionalQuantization optionalLow-bit quantization mentioned— | TensorRT-LLM all-reduce backend used— | — | Triton MoE kernel usedHPC-Ops attention backend optionalHPC-Ops fused MoE backend optional— | — | Tool-call parser used— |
| MiMo-V2.6-Flash core | — | DFlash coreFive-layer SWA MTP drafter coreDistribution-matched speculative-draft fine-tuning usedThroughput-based draft-block sizing usedEAGLE optionalSpeculative decoding optional— | — | Persistent per-dialogue-context KV caching coreAsynchronous KV-cache offloading usedHierarchical KV caching across HBM and pinned host memory used— | Dynamic activation scaling coreFP8 low-precision speculative draft computation used— | DeepEP used— | Greedy feasible-rank placement used— | — | — | Minimal, modular mini-harnesses for controlled harness diversity usedRequest timeouts and retries usedMulti-agent collaboration mentioned— |
| NVIDIA-Nemotron-3-Ultra-550B-A55B core | — | Recursive shared MTP-head drafting coreSpeculative decoding defaultBest-of-N selection usedEAGLE usedMulti-Token Prediction used— | Chain-of-thought reasoning usedGenerate-verify-refine usedThinking mode usedTurn budget capping usedTurn-aware prompting usedReasoning token-budget control optionaleffort parameters mentioned— | Chunked prefill defaultPrefix caching used8-bit Mamba cache quantization evaluatedPeriodic cache checkpointing evaluated— | Mixed-FP8 layers in an NVFP4 recipe coreNVFP4 routed-expert GEMM quantization corePer-tensor FP8 quantization coreSelective BF16 retention for sensitive layers coreNVFP4 KV-cache quantization defaultFour-Over-Six usedFP16 cache storage with stochastic rounding usedFP8 usedFP8 KV-cache quantization usedHeuristic mixed per-layer precision quantization usedMamba SSM-cache quantization usedPost-training quantization usedRandom Hadamard transform usedW4A16 quantization usedBlock-scaled INT8 quantization with stochastic rounding evaluatedEmpirical bits-per-element budget selection evaluatedFP8 E4M3 quantization evaluatedMax-based weight scaling evaluatedMSE-based weight-scale selection evaluated— | Prefill-decode disaggregation usedTopology-aware NVLink domain placement usedAttention data parallelism optionalLow-precision MoE combine optional— | Composite-key topology-aware GPU rank assignment usedNUMA binding for GPU-local CPU affinity usedNVLink domains as Ray custom resources used— | Marlin NVFP4 kernels usedMoE-side chunking used— | Discard-all context management usedSummary-based context compression evaluated— | Runtime command filtering usedTool-Integrated Reasoning used— |
| DeepSeek-V4-Flash core | — | — | Persistent reasoning history usedThink-tag response formatting usedThinking mode optional— | Customized heterogeneous KV-cache layout coreDynamic allocation of fixed-size state-cache pools usedKV-cache and sparse-attention-kernel co-design usedOn-disk KV-cache storage for shared-prefix reuse usedPeriodic cache checkpointing optionalZero SWA caching optional— | FP4 attention-indexer precision usedFP8 precision used— | — | — | Batch-invariant deterministic kernels coreDeepGEMM-based batch-invariant matrix multiplication usedDistributed shared memory for cross-SM attention exchange usedDual-kernel batch-invariant attention decoding usedFused mHC kernels usedHost Codegen usedPer-SM accumulation buffers with deterministic global summation usedSeparate split-K outputs with deterministic reduction usedToken-order preprocessing and cross-rank buffer isolation usedAvoiding split-K for batch invariance not used— | — | Agentic search usedRetrieval-Augmented Search usedXML-based tool-call schema with DSML token used— |
| DeepSeek-V3.2 used | — | Longest-thinking-trace selection usedMulti-stage candidate filtering usedTop-p and top-k sampling used— | Generate-verify-refine usedReasoning-enabled boolean toggle optionalParallel test-time compute scaling evaluatedParallel-fewest-step sampling evaluatedSerial test-time compute scaling through context management evaluatedThinking mode evaluated— | — | — | — | — | — | Context management method usedTest-time context management for extending token budgets usedTool-use reasoning-context persistence and selective deletion usedDiscard-75% evaluatedDiscard-all context management evaluatedTrajectory summarization and rollout re-initiation evaluated— | Cold-start prompting for reasoning in tool use usedJupyter Notebook code interpreter usedPrompt-enforced tool-call format usedThinking with tools used— |
| DeepSeek-V4-Flash-Vision-Exp default | — | Same-checkpoint target and draft weights defaultChat Prefix Completion optionalDSpark optional— | Thinking mode optional— | — | FP8 KV-cache quantization optional— | — | — | — | — | Compact TXT image-path notation optionalFunction calling optionalJSON mode optionalOpenAI-style JSON content blocks optional— |
| DeepSeek-V4-Pro core | — | — | Persistent reasoning history usedThink-tag response formatting usedThinking mode used— | Customized heterogeneous KV-cache layout coreDynamic allocation of fixed-size state-cache pools usedKV-cache and sparse-attention-kernel co-design usedOn-disk KV-cache storage for shared-prefix reuse usedPeriodic cache checkpointing optionalZero SWA caching optional— | FP4 attention-indexer precision usedFP8 precision used— | — | — | Batch-invariant deterministic kernels coreDeepGEMM-based batch-invariant matrix multiplication usedDistributed shared memory for cross-SM attention exchange usedDual-kernel batch-invariant attention decoding usedFused mHC kernels usedHost Codegen usedPer-SM accumulation buffers with deterministic global summation usedSeparate split-K outputs with deterministic reduction usedToken-order preprocessing and cross-rank buffer isolation usedAvoiding split-K for batch invariance not used— | — | Agentic search usedRetrieval-Augmented Search usedSandbox infrastructure (DSec) usedXML-based tool-call schema with DSML token used— |
| DeepSeek-V4-Pro-0813 optional | — | Chat Prefix Completion optionalDSpark optional— | Thinking mode optional— | — | — | — | — | — | — | Function calling optionalJSON mode optional— |
| FLUX 3 Action used | Pre-deployment robot-action safety checks used— | — | — | — | — | — | — | — | — | — |
| GLM-5.2 default | — | — | Thinking mode default— | Prefix caching usedRadixCache used— | Flex_AWQ_SSZ usedFP8 usedQuaRot usedW4A8 quantization used— | Attention data parallelism usedPrefill-decode disaggregation used— | Asynchronous inference and execution used— | Sparse Flash Attention used— | Discard-all context management usedHierarchical context management usedKeep-recent-k used— | — |
| GR00T N1.7 optional | Server-client policy inference optional— | — | — | — | — | — | — | — | — | — |
| Gemma 4 31B core | — | Multi-Token Prediction coreStandardized sampling configuration defaultClustered top-K projection usedDrafter-target KV-cache sharing used— | control-token-enabled thinking mode coreThinking mode coreConfigurable thinking mode controlled by the <|think|> token optionalconfigurable thinking/reasoning mode for LLM inference optionalStep-by-step thinking mode optional— | Key-as-value reuse in global attention layers usedKV cache sharing usedPrompt caching usedStateless in-memory prefix caching used— | Mobile quantization coreQuantization coreChannel-wise quantization usedEmbedding and KV-cache quantization usedStatic activation quantization usedTargeted 2-bit quantization usedQ4_0 optionalPost-training quantization not used— | — | Exacto routing optionalConcurrent request batching evaluated— | — | Excluding prior thinking from multi-turn history defaultModality-specific deployment optional— | Function calling coreSandboxed Python execution loop usednative function calling mentioned— |
| Inkling core | — | Multi-Token Prediction core— | Continuously controllable reasoning effort usedThinking mode used— | Prompt caching used— | Mixed-precision NVFP4 routed-expert quantization default— | Fused reduce-scatter/all-gather collectives used— | — | FA4 sheared-bias attention kernel used— | — | Python tool for visual tasks usedClaude Code skills for Tinker optional— |
| Kimi K3 core | — | Fused recurrent replay kernel coreMTP-to-EAGLE-3 draft-model fine-tuning used— | Always-on thinking mode coreChain-of-thought reasoning defaultconfigurable reasoning effort via reasoning_effort (low/high/max) defaultThinking mode defaultGeneration-prefix mode selection usedNatural-language reasoning-effort option message usedReasoning token-budget control usedStage-wise effort-budget curriculum used— | Fine-grained prefix hashing within cache pages coreKDA-aware prefix-cache management coreProjected-input caching for speculative KDA rollback coreShared-free-list cache allocation coreSparse hash-aligned KDA recurrent-state checkpoints coreUnified paged cache layout for KDA states and MLA KV coreWrite-back external KV-cache policy coreKDA with prefill cache usedKV-cache-aware placement of one-shot option messages used— | Offline weight-layout permutation coreMXFP4 weight quantization usedMXFP8 activation quantization used— | Zero-copy fused token permutation and unpermutation coreMoonEP token migration for load balancing unclear— | Cross-group cache-hit block pinning coreDual-cluster consistent-hash failover corePrefix-cache-aware session affinity scheduling coreRequest-class resource-budget admission control coreRuntime-signal-based rollout concurrency auto-throttling coreWorkload-aware routed-expert GEMM scheduling core— | Fused AttnRes merge and RMSNorm coreFused latent down-projection and MoE-router GEMM coreGPU kernel for online expert placement planning coreSeparate-stream shared-expert GEMM overlap coreSide-stream overlap for inter-block attention-residual computation coreSynchronization-free static-shape MoE execution coreWarpDecode token-centric routed-expert decoding coreFlashKDA usedKDA algorithm-system co-design used— | Preserved thinking history coreContext compaction used— | Function calling coreNatural-language instructions for request options corePeriodic sandbox snapshots coreConcurrent subagent orchestration usedCopy-on-write memory with page-cache optimization usedDynamic tool loading usedIndexed parallel tool calls usedmicroVM-isolated sandbox runtime for agentic AI usedOverlayBD shared-image sandbox launch stack usedTyped tool arguments usedVision in the loop usedXTML chat template usedAgentic tool calling mentioned— |
| Kolibri-1 core | — | Greedy decoding used— | Thinking mode coreChat-template thinking-mode switch defaultSystem-prompt reasoning-effort conditioning defaultLength-dependent completion-budget signal usedSoft probabilistic assignment of reasoning-effort labels used— | — | — | FP8 parameter quantization during transfer used— | Priority-based, topology-aware GPU job scheduling used— | — | — | Kolibri 1 reasoning parser coreMulti-step retrieval with grounded answering and abstention coreEgress allowlist and prompt hardening usedSearch-efficiency constraints usedUnprivileged-user sandboxes usedApplication-level output validation optionalHermes-style tool calling optionalIndependent agent review gate optional— |
| LFM3-1.7B default | Interleaved text-and-audio generation optional— | Greedy decoding usedTask-specific sampling-parameter presets optional— | Reasoning enabled by default default— | — | — | — | — | — | Remove prior-turn reasoning unless preserved defaultAutomatic prompt truncation usedInference-time vision-token budget scaling optional— | Application-managed tool execution usedDeferred tool loading usedRobot control through tool orchestration usedTemplate-based tool formatting usedRetrieval or search optional— |
| Laguna-S-2.1 core | — | DFlash optional— | Interleaved thinking coreMax thinking mode with self-set compute budget defaultThinking mode defaultPersistent reasoning history usedPer-request thinking control optionalReasoning effort configuration availability not used— | KV-cache reset on weight synchronization used— | AWQ INT4 weight-only quantization usedFP8 KV-cache quantization usedFP8 KV-cache quantization with amax-based scaling usedFP8 W8A8 quantization with dynamic activation scaling usedLayer-wise mixed INT4/INT8 quantization usedMoE-layer quantization usedSpinQuant R1 rotation preprocessing usedTRT-LLM inference optimization to NVFP4 on Blackwell usedOn-the-fly block-wise FP8 weight quantization not used— | — | Envoy proxy with custom orchestrator used— | — | Preserved thinking history used— | Current-turn-only reasoning-mode detection usedMulti-token streaming-delta handling usedXML-tagged tool-call format usedCheckpointing for rollback optionalTask-specification prompting mentioned— |
| LingBot-VLA 2.0 used | — | — | — | — | — | — | — | FlexAttention usedOperator fusion using torch.compile used— | — | — |
| MiMo-V2.5 used | — | Multi-Token Prediction usedSpeculative decoding usedEAGLE optional— | — | Chunked prefill usedRequest-level prefix cache usedRadixCache not used— | FP8 used— | DeepEP usedAttention data parallelism optional— | — | FlashAttention-3 used— | Discarding tool-call history usedMemory compression used— | Bash commands for context retrieval used— |
| MiMo-V2.5-Pro used | — | Multi-Token Prediction usedEAGLE optionalMulti-layer EAGLE optional— | — | — | — | DeepEP optional— | — | — | — | — |
| MiMo-V2.6-Pro core | — | DFlash coreFive-layer SWA MTP drafter coreDistribution-matched speculative-draft fine-tuning usedThroughput-based draft-block sizing used— | Chain-of-thought reasoning used— | Persistent per-dialogue-context KV caching coreAsynchronous KV-cache offloading usedHierarchical KV caching across HBM and pinned host memory used— | Dynamic activation scaling coreFP8 low-precision speculative draft computation used— | DeepEP used— | Greedy feasible-rank placement used— | — | — | Minimal, modular mini-harnesses for controlled harness diversity usedRequest timeouts and retries used— |
| MiniMax-M3 core | — | — | Test-time compute scaling usedAdaptive reasoning optionalThinking mode optional— | Block-based KV cache coreAutomatic cache support defaultPre-scheduled tile chunking used— | — | — | Host-side scheduling optimization used— | CUDA Graph capture and replay usedDynamic load balancing usedExp-free TopK kernel usedFlashAttention usedPersistent kernel usedTwo-phase forward usedFP8 GEMM evaluated— | — | Producer–Verifier adversarial harness loop usedReAct Toolbelt used— |
| Mistral-Large-4 core | — | DSpark coreRecurrent Markov draft head coreThree-layer dense MLA speculative drafter coreThree-token speculative draft depth coreBlock verification usedRecurrent conditioning head usedTemperature and nucleus sampling configuration usedUnique mask tokens used— | Reasoning cannot be disabled defaultThinking mode default— | Cross-layer KV-cache sharing corePrefix caching defaultYou Only Cache Once (YOCO) usedDecode context parallelism optional— | FP8 latent KV-cache quantization defaultAbs-max quantization usedFP8 usedMixed-precision NVFP4 routed-expert quantization usedNative FP8 KV cache with offloading usedOCP MXFP4 quantization usedSFT-mixture calibration usedLayer-by-layer FP8-to-MXFP4 expert re-quantization at load optional— | Disaggregated serving across 4+4 GPUs usedTensor and expert parallelism used— | Metric-driven inference-fleet autoscaling usedSession-sticky routing to data-parallel ranks usedProvider routing mentioned— | Breakable CUDA graphs defaultFA4 blockwise attention usedKernel fusion usedPersistent FlyDSL fused MoE decode kernel optionalSingle-launch fusion of attention, indexer, routing, experts, and all-reduces optional— | — | Strict JSON Schema output coreMicroVM-based sandbox isolation usedMinimal tool-set design usedSystem-prompt guidance for tool use usedTurn-end hook used— |
| MolmoAct 2 core | Pre-deployment robot-action safety checks optionalReusable-cache optimization optional— | Action-chunk execution usedAutoregressive discrete action-token inference optional— | Adaptive reasoning core— | Cached depth-token replay for unchanged cells usedPreallocated static KV cache used— | — | — | Caching invariant action-expert intermediates usedEager adaptive depth scheduling usedStatic-shape compilation used— | CUDA Graph capture and replay used— | Adaptive depth-token prediction with cached unchanged cells coreCosine similarity used— | Visual trace steering coreInteractive natural-language steering usedMatching training and inference prefixes usedPrompt descriptors for embodiment and control semantics usedPrompting Molmo to locate the robot gripper usedTest-time action steering through language or visual traces optionalTrajectory editing optional— |
| NVIDIA-Nemotron-3.5-Lightning-30B-A3B core | Platform- and workload-specific quantization and speculative-decoding tuning used— | Speculative decoding coreTop-K candidate lattice coreBest-of-N selection usedLiLiCorr single-pass correlation head usedMulti-Token Prediction usedSemi-autoregressive drafting with sequential Markov correction usedStandalone parallel speculative drafters usedDFlash optionalDSpark optionalProbabilistic draft sampling optionalConcurrency-aware draft-length tuning evaluatedConfidence head not used— | Reasoning token-budget control usedParallel test-time compute scaling evaluated— | Checkpointing replay for Mamba recurrent-state cache usedHybrid-state caching usedInternal prefill checkpointing usedMamba checkpointing for speculative-decoding replay optional— | Backbone-coupled MTP KV-cache precision coreShared FP8 lm_head for target verification and MTP drafting defaultAggressive mixed-precision quantization recipe usedBF16 MTP attention weights with FP8 K/V BMM quantizers usedFP8-or-higher lm_head precision usedGradient-based per-layer mixed-precision quantization search usedMSE-based weight-scale selection usedNVFP4 post-training quantization usedNVFP4 W4A4 quantization of MTP experts usedNVFP4 weight quantization with BF16 activations usedParameter alignment for quantization usedPer-block FP8 quantization for attention Q/K/V projections usedPost-training calibration sample count usedPost-training quantization usedQuantization-aware distillation usedSeparate MTP quantization and backbone splicing usedTask-loss-gradient-weighted quantization sensitivity estimation usedWeight-only post-training quantization for parallel drafters usedNVFP4 quantization optionalSeparate target and MTP lm_head precision optionalBF16 expert activations evaluatedBlock-scaled NVFP4 KV-cache quantization evaluatedFP4 quantization of Mamba output-projection weights evaluatedFP8 or NVFP4 KV-cache quantization evaluatedIncrease MoE expert precision and fine-tune on RL rollouts evaluatedLinear E2M1 residual correction evaluatedMXFP8 block-scaled attention projection quantization evaluatedPer-tensor FP8 quantization evaluatedPromote expert weights from NVFP4 to FP8 evaluatedQuantization of the parallel-drafter language-model head evaluatedRestricted-search MSE weight-scale selection evaluatedRotation or higher precision for Mamba input activations evaluatedSensitivity-driven mixed-precision quantization evaluatedTwo-pass FP4 weight quantization with residual correction evaluatedW4A16 quantization of Mamba output-projection weights evaluatedHigher-precision activations for Mamba output projection mentionedRotation-based quantization mitigation mentionedMixed-precision key/value cache quantization not usedW4A4 quantization not used— | Disaggregated serving evaluated— | KV-cache-aware routing usedSession affinity routing used— | GPU-kernel optimization usedMulti-wave KV-cache splitting optionalPer-quantization-scheme GEMM backend selection optional— | — | Model routing by planning and execution role coreBash computer-use agent usedParallel agent execution with self-verification used— |
| NVIDIA-Nemotron-3.5-Super-VL-120B-A12B core | Platform- and workload-specific quantization and speculative-decoding tuning used— | Speculative decoding coreTop-K candidate lattice coreLiLiCorr single-pass correlation head usedSemi-autoregressive drafting with sequential Markov correction usedStandalone parallel speculative drafters usedDFlash optionalDSpark optionalLiLiCorr optionalMTP draft-length configuration optionalProbabilistic draft sampling optionalConfidence head not used— | Reasoning enabled by default defaultReasoning token-budget control optionalThinking mode optionalParallel test-time compute scaling evaluated— | Attention-cache and Mamba-state preservation coreHybrid-state caching corePrefix caching corePrefix reuse coreCheckpointing replay for Mamba recurrent-state cache usedInternal prefill checkpointing usedMamba checkpointing for speculative-decoding replay usedKV-cache management optionalKV-cache offloading evaluated— | Backbone-coupled MTP KV-cache precision coreMixed-precision inference coreNative 2-bit quantization coreShared FP8 lm_head for target verification and MTP drafting defaultAggressive mixed-precision quantization recipe usedBF16 MTP attention weights with FP8 K/V BMM quantizers usedComponent-wise precision selection usedFP8-or-higher lm_head precision usedGradient-based per-layer mixed-precision quantization search usedMSE-based weight-scale selection usedNVFP4 post-training quantization usedNVFP4 W4A4 quantization of MTP experts usedNVFP4 weight quantization with BF16 activations usedParameter alignment for quantization usedPer-block FP8 quantization for attention Q/K/V projections usedPlatform-dependent W4A4/W4A16 quantization usedPost-training calibration sample count usedPost-training quantization usedQuantization-aware distillation usedSeparate MTP quantization and backbone splicing usedTask-loss-gradient-weighted quantization sensitivity estimation usedWeight-only post-training quantization for parallel drafters usedNVFP4 weight-and-activation quantization optionalSeparate target and MTP lm_head precision optionalBF16 expert activations evaluatedBlock-scaled NVFP4 KV-cache quantization evaluatedFP4 quantization of Mamba output-projection weights evaluatedFP8 or NVFP4 KV-cache quantization evaluatedIncrease MoE expert precision and fine-tune on RL rollouts evaluatedLinear E2M1 residual correction evaluatedMXFP8 block-scaled attention projection quantization evaluatedPer-tensor FP8 quantization evaluatedPromote expert weights from NVFP4 to FP8 evaluatedQuantization of the parallel-drafter language-model head evaluatedRestricted-search MSE weight-scale selection evaluatedRotation or higher precision for Mamba input activations evaluatedSensitivity-driven mixed-precision quantization evaluatedTwo-pass FP4 weight quantization with residual correction evaluatedW4A16 quantization of Mamba output-projection weights evaluatedHigher-precision activations for Mamba output projection mentionedRotation-based quantization mitigation mentionedMixed-precision key/value cache quantization not usedW4A4 quantization not used— | Disaggregated serving optional— | KV-cache-aware routing usedSession affinity routing usedAsynchronous inference and execution optionalProvider failover routing optional— | GPU-kernel selection coreGPU-kernel optimization usedMulti-wave KV-cache splitting optionalPer-quantization-scheme GEMM backend selection optional— | Context reuse core— | Parallel agent execution with self-verification used— |
| OpenVLA / OpenVLA-OFT default | Remote VLA inference server used— | Greedy decoding usedSecond-most-likely action querying usedSpeculative decoding mentionedDynamic action querying not used— | — | — | BF16 inference defaultQuantization optional4-bit quantization evaluated8-bit and 4-bit quantization evaluated8-bit quantization evaluatedQuantized and reduced-precision inference evaluated— | — | — | FlashAttention optionalFlashAttention-2 optional— | — | — |
| Qwen3.5-397B-A17B default | — | MTP-1 speculative decoding usedPresence penalty tuning usedMulti-Token Prediction optionalNEXTN speculative decoding optionalTask-specific sampling-parameter presets optional— | Reasoning enabled by default defaultThinking mode defaultDisable reasoning via enable_thinking optionalTask- and mode-specific sampling parameters optionalTask-appropriate maximum output length optionalQwen3 soft thinking switch not used— | Chunked prefill usedLanguage-model-only serving mode usedPrompt caching usedPrefix caching optional— | FP8 usedNVFP4 quantization used— | Data-parallel vision encoding usedTensor parallelism optional— | — | — | Excluding prior thinking from multi-turn history defaultContext folding usedDiscard-all context management evaluated— | Talker system prompt for voice characteristics usedMCP configuration for tool definition optionalTool calling with the qwen3_coder parser optional— |
| Qwen3.6-35B-A3B default | — | Multi-Token Prediction optionalTask-specific sampling-parameter presets optional— | Interleaved thinking defaultThinking mode defaultPersistent reasoning history usedDisable reasoning via enable_thinking optional— | Chunked prefill usedPrefix caching usedPrompt caching usedLanguage-model-only serving mode optionalMamba prefix caching in align mode evaluated— | FP8 KV-cache quantization usedFP8 optionalModelOpt NVFP4 re-quantization optionalNVFP4 quantization optional— | — | Asynchronous inference and execution used— | CUDA graph capture size reduction used— | Preserved thinking history used— | Automatic tool choice usedQwen3 XML tool-call parser usedMCP configuration for tool definition optional— |
| Qwen3.8-Flash-Next core | — | NEXTN speculative decoding usedMulti-Token Prediction optional— | Reasoning enabled by default default— | Overlapped host-memory prefetching used— | Fine-grained FP8 quantization optional— | Tensor parallelism defaultExpert parallelism optional— | Continuous batching optional— | Shape-aware kernel dispatch for Hyper-Connection coreSparse pinned-host offload defaultCUDA Graph capture and replay usedFused QSA attention and KL-loss kernel usedFusing activation, gating, and reduction into GEMM epilogues usedHidden-dimension split Combine kernel usedSplit-K CuTe GEMM used— | Preserved thinking history default— | — |
| SmolVLA core | Chunk fusion core— | Action-chunk execution evaluated— | — | — | — | — | Asynchronous inference and execution coreThreshold-triggered inference core— | — | — | — |
| Step-3.7-Flash used | — | EAGLE optionalMulti-layer EAGLE optional— | Reasoning parser usedStep3p5 reasoning parser usedThinking mode optional— | — | FP16 multimodal projector usedFP8 usedFP8 KV-cache quantization usedGGUF usedModelOpt FP4 quantization usedModelOpt NVFP4 re-quantization usedNVFP4 quantization usedNVIDIA ModelOpt quantization usedIQ4_XS optionalQ3_K_L optionalQ4_K_S optional— | — | Asynchronous inference and execution used— | TensorRT-LLM multi-head attention backend usedFlashAttention-4 optional— | — | Automatic tool choice usedPython tool for visual tasks usedstep3p5 tool-call parser usedTool-call parser usedVisual Search Tool usedAdvisor strategy optional— |
| VLA Foundry (Foundry-VLA-1.7B) used | — | — | — | — | — | — | — | — | Temporal action chunking with past-context conditioning used— | — |
| X-VLA used | — | — | — | — | — | — | — | — | — | Embodiment-aware language prompts used— |
| Xiaomi-Robotics-1 core | — | — | — | — | — | — | Asynchronous inference and execution coreAction-chunk timestep alignment used— | FlashAttention-2 used— | Four-step observation history used— | — |
| gpt-oss-120b core | — | Grammar-constrained decoding usedBest-of-N selection evaluated— | Full chain-of-thought output coreThinking mode used— | — | BF16 inference usedMXFP4 block-scaled tensor packing usedMXFP4 weight quantization used— | Tensor parallelism for MoE layers used— | — | CUDA Graph capture and replay usedExpert-optimized Triton kernels usedOptimized Triton MoE kernel with MXFP4 support used— | Excluding prior thinking from multi-turn history used— | Assistant output channels coreChain-of-thought in the analysis channel coreDeveloper message format coreHarmony format coreHarmony function-calling format coreHarmony tool-call message format coreRole-based instruction hierarchy coreBrowser tool usedBuilt-in tool declaration in the system message usedCommentary-channel preambles usedDeveloper-defined function schemas usedHarmony history end-token normalization usedHarmony system message format usedInterleaved tool calls in chain-of-thought usedJSON Schema response formats usedPython execution tool in chain-of-thought usedRetaining chain-of-thought across tool calls usedStateful Python tool usedStateless Python tool reference implementation usedTool message format usedTypeScript-like function schema syntax usedStreamableParser optional— |
| π0.5 core | — | Greedy decoding defaultTemperature sampling for bimanual tasks optional— | Hierarchical subtask inference coreHigh-level language policy for task decomposition used— | Robot-state KV caching across sampling steps corePrefix key-value caching during action inference used— | — | — | — | — | — | GPT-4 high-level policy evaluated— |