Thinking Machines Lab
Inkling
What Inkling’s own documents say it is built from — every method with the sentence that describes it.
- Parameters
- 975B / 41B active
- Context
- 1M
- Attention
- sliding/global hybrid66 layers, mostly sliding-window (512) with every 6th layer global GQA, QK-norm
- Experts
- 256 routed + 2 shared, top-6
- Input
- text, image, audio, video
- Licence
- Apache 2.0
- Released
- 2026-07-15
Against the consensus recipe
Its documents state 6 of the 18 methods the field agrees on.
Data curation
- Benchmark decontaminationnot in its documents
- Synthetic data generationused
- Corpus deduplicationnot in its documents
Training objective
- Multi-Token Predictionnot in its documents
- Knowledge insulationnot in its documents
- Quantization-aware distillationnot in its documents
Model architecture
- Mixture of expertscore
- Multi-Token Predictionnot in its documents
- Grouped-query attentionused
Optimization
- Muonnot in its documents
- Expert parallelismnot in its documents
- Quantization-Aware Trainingnot in its documents
Post-training
- Supervised fine-tuningnot in its documents
- Asynchronous reinforcement learningcore
- Multi-Teacher On-Policy Distillationnot in its documents
Inference & serving
- Thinking modeused
- Multi-Token Predictioncore
- FP8not in its documents
Checked in its code
Its code shows 9 of the 19 architecture features checked — 4 of them are not stated in its documents.
Sources say what a lab chose to describe; the modeling code says what the checkpoint runs. Each feature below was put to a code verifier, and only the config values and source lines it cited that were found in the files are shown, linked to the line.
Read from vLLM (__init__.py, model.py, configs.py, attention.py, layernorm.py, fa4_rel_attention.py, qkvr_prep.py, sconv_swa_attn.py, sconv.py, short_conv.py, logits_processor.py, mlp.py, moe.py, lamport.py, norm.py, mtp.py) and config.json at revision 828496eeae, by anthropic/claude-sonnet-5 on 2026-11-07.
| Feature | Code | Its documents | Verdict |
|---|---|---|---|
| Grouped-query attention | in its code | adopted | Both full and local attention layers project queries with more heads than K/V heads (64 query heads vs 8 KV heads full, 64 vs 16 local)… show evidenceBoth full and local attention layers project queries with more heads than K/V heads (64 query heads vs 8 KV heads full, 64 vs 16 local), implementing grouped-query attention. text_config.num_attention_heads = 64text_config.num_key_value_heads = 8 |
| Multi-head latent attention | not in its code | not stated | No kv_lora_rank/q_lora_rank-style latent down/up projection exists; attention here is a GQA relative-position design with separate… show the whole verdictNo kv_lora_rank/q_lora_rank-style latent down/up projection exists; attention here is a GQA relative-position design with separate per-head K/V projections, not MLA. |
| Sliding-window attention | in its code | adopted | Layers marked local (local_layer_ids) use a sliding window of size sliding_window_size=512 implemented via the FA4 window_size and… show evidenceLayers marked local (local_layer_ids) use a sliding window of size sliding_window_size=512 implemented via the FA4 window_size and SlidingWindowSpec KV cache. text_config.sliding_window_size = 512text_config.local_layer_ids = [0, 1, 2, 3, 4, 6, 7, 8, 9, 10, 12, 13, 14, 15, 16, 18, 19, 20, 21, 22, 24, 2… |
| Interleaved sliding-window and global attention | in its code | adopted | local_layer_ids selects most layers as sliding-window (swa_*) while layers not listed (e.g. 5, 11, 17...) use full attention, giving a… show evidencelocal_layer_ids selects most layers as sliding-window (swa_*) while layers not listed (e.g. 5, 11, 17...) use full attention, giving a fixed interleaving of local and global attention layers. text_config.local_layer_ids = [0, 1, 2, 3, 4, 6, 7, 8, 9, 10, 12, 13, 14, 15, 16, 18, 19, 20, 21, 22, 24, 2…text_config.num_hidden_layers = 66 |
| Indexer-selected sparse attention (DeepSeek Sparse Attention) | not in its code | not stated | No indexer-based top-k token selection mechanism (no index_topk/index_n_heads/index_head_dim) is present anywhere in the attention code. |
| Linear-attention or state-space layers alongside full attention | not in its code | not stated | All token-mixing layers use the same relative-position softmax attention (full or sliding-window variants); there are no linear-attention… show the whole verdictAll token-mixing layers use the same relative-position softmax attention (full or sliding-window variants); there are no linear-attention or SSM mixer layers in this model. |
| Gated DeltaNet layers | not in its code | not stated | No gated delta-rule recurrent mixer exists; the only non-attention mixing component is a short depthwise convolution (sconv), not a… show the whole verdictNo gated delta-rule recurrent mixer exists; the only non-attention mixing component is a short depthwise convolution (sconv), not a delta-rule gated state. |
| Mamba-2 layers | not in its code | not stated | There are no Mamba-2 SSD blocks; config only carries short-conv (sconv) parameters, not SSM state sizes or per-head decay. |
| Learnable attention sink | not in its code | not stated | No learned per-head sink logit is added to the attention softmax denominator anywhere in the attention or kernel code. |
| Gated attention output | not in its code | not stated | The attention output is passed directly to the output projection (wo_ud) with no sigmoid gate multiplication applied to it. show evidence |
| QK normalization code only | in its code | not stated | Queries and keys are each passed through per-head RMSNorm (q_norm/k_norm) before the attention dot product. show evidence |
| Partial RoPE | not in its code | not stated | The model uses a learned relative-position bias (d_rel/rel_extent projected logits added to attention scores) rather than rotary position… show the whole verdictThe model uses a learned relative-position bias (d_rel/rel_extent projected logits added to attention scores) rather than rotary position embeddings at all, so there is no partial-RoPE mechanism; no rotary application appears anywhere in the attention code. |
| YaRN RoPE scaling | not in its code | not stated | There is no rope_scaling key in the config and no RoPE scaling code of any kind; positions are encoded via a learned relative-bias… show the whole verdictThere is no rope_scaling key in the config and no RoPE scaling code of any kind; positions are encoded via a learned relative-bias projection, not rotary embeddings. |
| Layers without positional encoding (NoPE) | code unclear | not stated | The model has no RoPE at all (it uses a learned relative-position bias instead), so there is no distinction between layers with and… show the whole verdictThe model has no RoPE at all (it uses a learned relative-position bias instead), so there is no distinction between layers with and without positional encoding visible in this code; this check does not clearly apply and the files don't show any per-layer toggle for positional encoding. |
| Mixture of experts | in its code | adopted | Feed-forward layers beyond dense_mlp_idx route tokens to a learned sigmoid gate selecting top-6 of 256 routed experts via… show evidenceFeed-forward layers beyond dense_mlp_idx route tokens to a learned sigmoid gate selecting top-6 of 256 routed experts via InklingMoE/FusedMoE. text_config.n_routed_experts = 256text_config.num_experts_per_tok = 6 |
| Shared expert | in its code | adopted | MoE layers include 2 shared 'sink' experts that every token passes through in addition to the routed experts, enabled by… show evidenceMoE layers include 2 shared 'sink' experts that every token passes through in addition to the routed experts, enabled by n_shared_experts=2 and shared_expert_sink=true. text_config.n_shared_experts = 2text_config.shared_expert_sink = true |
| Auxiliary-loss-free load balancing (selection bias) code only | in its code | not stated | The gate adds a learned per-expert bias to the sigmoid routing scores used only for top-k selection, controlled by use_gate_bias=true… show evidenceThe gate adds a learned per-expert bias to the sigmoid routing scores used only for top-k selection, controlled by use_gate_bias=true (DeepSeek-style selection-bias load balancing). text_config.use_gate_bias = true |
| Multi-token prediction layers code only | in its code | not stated | The config declares 8 next-next-token prediction (MTP) depth layers, and the model ships an InklingMTP module that builds and runs these… show evidenceThe config declares 8 next-next-token prediction (MTP) depth layers, and the model ships an InklingMTP module that builds and runs these draft transformer blocks. mtp_config.num_nextn_predict_layers = 8 |
| SwiGLU feed-forward code only | in its code | not stated | Dense and shared-expert MLPs compute SiLU(gate(x)) * up(x) (silu_and_mul / torch.nn.functional.silu gating), matching SwiGLU, consistent… show evidenceDense and shared-expert MLPs compute SiLU(gate(x)) * up(x) (silu_and_mul / torch.nn.functional.silu gating), matching SwiGLU, consistent with hidden_act='silu' default (not overridden to gelu in this config). |
filed at the root 2
“Inkling uses a regular Transformer decoder rather than the hybrid Mamba-Transformer approach used by Nemotron 3 Ultra.”
“We also apply short convolutions at two points — after the key and value projections in each attention layer, and on the attention and MLP…”
token mixer 4
“Each decoder layer applies short kernel-4 convolutions after the key and value projections and on the attention and MLP branch outputs.”
softmax attention 2
sliding window attention 1
“Of the 66 decoder layers, 55 use local attention with a small 512-token window.”
grouped-query attention 1
“the use of conventional GQA rather than MLA or a recurrent hybrid stack suggest that raw decoding speed may not be Inkling’s main advantage”
hybrid layer stacking 1
“Attention is a hybrid of local and global layers.”
channel mixer 5
mixture of experts 5
“A 66-layer decoder-only transformer with a sparse Mixture-of-Experts (MoE) feed-forward backbone”
“Inkling uses a sigmoid-based router with an auxiliary-loss-free load-balancing bias.”
expert routing 1
“each token is routed to 6 of 256 experts, plus 2 shared experts active on every token”
positional encoding 2
“Inkling replaces RoPE with relative attention: a learned relative-position term added to the pre-softmax logits.”
“In the 11 global layers, the released implementation applies the learned bias only over the preceding 1,024 tokens.”
normalization & residual 1
“An additional RMSNorm directly after the token embedding layer.”
multimodal architecture 5
“The model is natively multimodal — images and video are encoded via a hierarchical patch encoder, and audio via discrete token encoding…”
“audio via discrete token encoding”
“images and video are encoded via a hierarchical patch encoder”
“with all modalities projected into a shared hidden space and processed jointly by the decoder”
“We opted for an encoder-free architecture for audio and vision inputs, consistent with the interaction model design.”
optimizer 1
“We trained Inkling with a hybrid optimization strategy — Muon for large matrix weights, Adam for other parameters — and hyperparameter…”
training precision 1
“The NVFP4 checkpoint offers a quantized alternative that reduces the aggregated VRAM requirement to at least 600 GB.”
training stability 1
“We coupled the weight decay strength to the square of the learning rate, which we found kept the overall size of the model weights stable…”
training parallelism 1
“sconv-aware TP sharding (each rank stores only its channel shard of the sconv cache)”
filed at the root 2
“The training data curation process includes cleaning, processing, and modifying datasets.”
“may include deduplication and filtering to remove junk or other low-quality data, or to advance safety or other objectives”
synthetic data 2
“synthetically generated or augmented”
“we randomized the tool set and schema during training to reduce sensitivity to any particular one.”
filed at the root 1
“We post-trained Inkling on a broad distribution of math, agentic code & tool use, audio, image, chat, and safety domains.”
supervised fine-tuning 3
“create_lora_training_client”
“To bootstrap post-training, we ran an initial SFT on synthetic data generated by open-weights models including Kimi K2.5.”
“We trained Inkling to an internal spec of safe model behavior across all modalities.”
reinforcement learning algorithm 3
“before large-scale reinforcement learning took over”
“We trained for calibration with RL against proper scoring rules on a large corpus of resolved real-world questions.”
“We did RL with two automated graders: a rubric grader and claims grader.”
reward modelling 1
“The largest is short-form factual QA with abstention-aware rewards: answering only pays off when the model is likely to be right, so the…”
policy distillation 1
“it used other open-weight models — including Moonshot AI’s Kimi K2.5 — to help generate some of its early post-training data”
rollout & RL infrastructure 1
“We relied on large-scale asynchronous RL to shape model behavior and improve its reasoning and overall performance.”
decoding strategy 1
“8 chained MTP heads (single-layer full-attention transformers with a dense MLP, all BF16) enable up to 9 tokens per forward step for…”
reasoning control 2
“lets users dial 'thinking effort' up or down when they want to trade for speed”
“We specified the model’s effort level on different samples by changing the system message and adjusting the per-token cost. This caused…”
KV cache management 1
“The chart below shows the average price customers are actually paying after prompt caching.”
inference quantization 1
“Only the routed experts are quantized to NVFP4; shared experts and the qkvr linears remain BF16.”
serving parallelism 1
“low-latency fused reduce-scatter / all-gather collectives built on FlashInfer's Lamport-protocol all-reduce”
inference kernel 1
“the FA4 sheared-bias attention kernel for relative attention”
agentic scaffolding 2
“During inference, Inkling can also leverage a Python tool to support image understanding through operations such as zooming and cropping…”
“Tinker Cookbook ships with Claude Code skills that teach Claude how to use the Tinker API.”
infrastructure service 1
“OpenRouter monitors every provider continuously and automatically retries on the next-best provider when one returns an error.”
filed at the root 2
“refusal-suppressed variants intended to estimate latent capability with safeguards removed”
“We document our exact settings alongside all reported scores.”
evaluation harness 2
“Tinker Cookbook includes a benchmark framework for evaluating trained models”
“Benchmarks can also serve as inline training evaluators via `BenchmarkEvaluator`.”
human & real-world evaluation 3
“including through multi-turn, open-ended external red-teaming designed to surface issues that only emerge over longer conversations”
“For CBRN and cyber, we assessed knowledge and procedural uplift through internal evaluations, external testing, and refusal-suppressed…”
“We conducted safety evaluations ahead of release, spanning both everyday human-AI interaction and dangerous-capability testing.”
unfiled 18
“Each layer applies four window-4 sconv modules (on attention keys, values, output, and MoE output).”
“OpenRouter's API is OpenAI-compatible — most SDKs work by just swapping the base URL.”
“benchmarked against public frontier models”
“Inkling is designed to give calibrated answers, including flagging uncertainty rather than guessing”
“something for organizations to fine-tune themselves through Tinker”
“It's Apache-2.0 licensed, and looks competitive with the open weight models coming out of China”
“registers pre-commit hooks that run `ruff` formatting and linting on every commit. CI enforces these checks on all pull requests.”
“pre-trained Inkling from scratch”
“OpenRouter routes your request to one of them based on the routing mode you pick — Balanced (price + speed), Nitro (fastest), or Exacto…”
“Finally, we trained Inkling to answer directly on topics that may be subject to censorship.”
“a combination of qualities makes it a good open-weights base for customization: multimodal capabilities, efficient thinking, and…”
7 more mentioned or evaluated
“AI designed to listen and speak (and even interrupt) instead of stop and wait as with typical chatbots”
“Implement additional safeguards — such as content filtering, rate limiting, and monitoring — at the application layer”
“are best addressed with defense-in-depth rather than relying on the model’s refusals alone”
“are best addressed with defense-in-depth rather than relying on the model's refusals alone. Common downstream moderation tools, such as…”
“apply appropriate human oversight and review for outputs used in high-stakes or safety-critical contexts”
“Common downstream moderation tools, such as Llama Guard, are compatible with Inkling and can be layered around the model to catch…”
“Common downstream moderation tools, such as Llama Guard, are compatible with Inkling and can be layered around the model”
Further reading
Picked by hand, not extracted: where to read more, not evidence for anything on this page.
- Inkling architecture card — LLM Architecture Gallery explainer sebastianraschka.com
Sources
The curated source list for this model. "not fetched" means the URL is recorded but its text was not read in the current run.
| Source | Kind | Publisher | Status |
|---|---|---|---|
| thinkingmachines.ai/news/introducing-inkling | official blog | Thinking Machines Lab | read |
| thinkingmachines.ai/model-card/inkling | model card | Thinking Machines Lab | read |
| huggingface.co/thinkingmachines/Inkling | model card | Thinking Machines Lab | read |
| huggingface.co/thinkingmachines/Inkling-NVFP4 | model card | Thinking Machines Lab | read |
| github.com/thinking-machines-lab/tinker-cookbook | code repo | Thinking Machines Lab | read |
| recipes.vllm.ai/thinkingmachines/Inkling | vendor docs | vLLM | read |
| openrouter.ai/thinkingmachines/inkling | vendor docs | OpenRouter | read |
| sebastianraschka.com/blog/2026/inkling-architecture-benchmark-notes.html | third party analysis | Sebastian Raschka | read |
| artificialanalysis.ai/articles/thinking-machines-has-released-inkling-t… | third party analysis | Artificial Analysis | read |
| simonwillison.net/2026/Jul/16/inkling | third party analysis | Simon Willison | read |
| techcrunch.com/2026/07/15/thinking-machines-amps-up-its-bet-against-one… | news | TechCrunch | read |