Model techniques map
ModelsLaguna-S-2.1

Poolside · released 2026-07-21

Laguna-S-2.1

What Laguna-S-2.1’s own documents say it is built from — every method with the sentence that describes it.

OpenRouterHugging Face

Curator’s note118B total / 8B active MoE for agentic coding: 48 layers in a 1:3 global-to-sliding ratio (12 global, 36 SWA at window 512), 256 routed experts (top-10) + 1 shared, token-choice routing with softplus gating, grouped-query attention with per-head softplus output gating, 1M context, OpenMDW-1.1. RL post-training in FP8 over 409k agentic and non-agentic task environments, and a trained DFlash draft model for speculative decoding. First non-Chinese, non-US-hyperscaler lab in the map. The family technical report covers Laguna M.1 / XS.2, not S 2.1 itself; full evaluation trajectories are published at trajectories.poolside.ai.

Against the consensus recipe

Its documents state 5 of the 15 methods the field agrees on.

The recipe from the overview: the methods the most labs adopt, per stage. A missing one is something the documents do not say, not something the model lacks.

  1. Data curation

  2. Training objective

  3. Model architecture

  4. Optimization

  5. Post-training

  6. Inference & serving

Checked in its code

Its code shows 11 of the 19 architecture features checked — 8 of them are not stated in its documents.

Documents say what a lab chose to describe; the modeling code says what the checkpoint runs. Each feature below was put to a code verifier, and only the config values and source lines it cited that were found in the files are shown, linked to the line.

Read from Hugging Face Transformers (modeling_laguna.py, configuration_laguna.py) and config.json at revision 0f57314083, by anthropic/claude-sonnet-5 on 2026-09-25.

FeatureCodeIts documentsEvidence
Grouped-query attention in its code core

Each layer's queries (48 or 72 heads) share 8 key/value heads via repeat_kv-based grouped-query attention.

num_attention_heads = 48num_key_value_heads = 8
self.num_key_value_groups = self.num_heads // config.num_key_value_heads
modeling_laguna.py · L345
key_states = repeat_kv(key, module.num_key_value_groups)
    value_states = repeat_kv(value, module.num_key_value_groups)
modeling_laguna.py · L320–321
Multi-head latent attention not in its codenot stated

Attention uses separate q/k/v projections sized by head_dim and num_key_value_heads with no low-rank latent cache, so MLA is not implemented.

self.q_proj = nn.Linear(config.hidden_size, self.num_heads * self.head_dim, bias=config.attention_bias)
        self.k_proj = nn.Linear(
            config.hidden_size, config.num_key_value_heads * self.head_dim, bias=config.attention_bias
        )
modeling_laguna.py · L350–353
Sliding-window attention code only in its codenot stated

Layers marked sliding_attention in layer_types use sliding_window=512 to restrict attention to a local window.

sliding_window = 512layer_types = ["full_attention", "sliding_attention", "sliding_attention", "sliding_attenti…
self.is_local_attention = config.layer_types[layer_idx] == "sliding_attention"
        self.sliding_window = config.sliding_window if self.is_local_attention else None
modeling_laguna.py · L360–361
Interleaved sliding-window and global attention code only in its codenot stated

layer_types interleaves one full_attention layer followed by three sliding_attention layers repeatedly across all 48 layers.

layer_types = ["full_attention", "sliding_attention", "sliding_attention", "sliding_attenti…
mask_creation_functions = {
                "full_attention": lambda: create_causal_mask(**mask_kwargs),
                "sliding_attention": lambda: create_sliding_window_causal_mask(**mask_kwargs),
            }
modeling_laguna.py · L560–563
Indexer-selected sparse attention (DeepSeek Sparse Attention) not in its codenot stated

No indexer-based top-k token selection mechanism appears anywhere in the attention or MoE code.

Linear-attention or state-space layers alongside full attention not in its codenot stated

All decoder layers use the same LagunaAttention softmax attention class; there is no linear-attention or SSM mixer variant.

self.self_attn = LagunaAttention(config, layer_idx, config.num_attention_heads_per_layer[layer_idx])
modeling_laguna.py · L427
Gated DeltaNet layers not in its codenot stated

No delta-rule recurrent state update or decay gate code exists; all layers are standard softmax attention.

Mamba-2 layers not in its codenot stated

No selective state-space (SSD/Mamba-2) block implementation is present in the modeling file.

Learnable attention sink not in its codenot stated

The attention softmax and mask construction have no extra learned sink logit column.

attn_weights = nn.functional.softmax(attn_weights, dim=-1, dtype=torch.float32).to(query.dtype)
modeling_laguna.py · L327
Gated attention output code only in its codenot stated

Attention output is multiplied by a softplus (not sigmoid, but input-dependent) gate computed from hidden_states via g_proj, either per-head or per-element based on config.gating.

gating = "per-head"
gate = F.softplus(self.g_proj(hidden_states).float()).to(attn_output.dtype)
        if self.gate_per_head:
            attn_output = (attn_output.view(*input_shape, -1, self.head_dim) * gate.unsqueeze(-1)).view(
                *input_shape, -1
            )
modeling_laguna.py · L411–415
QK normalization code only in its codenot stated

Queries and keys are each normalized per head with LagunaRMSNorm before rotary and the attention dot product.

self.q_norm = LagunaRMSNorm(self.head_dim, eps=config.rms_norm_eps)
        self.k_norm = LagunaRMSNorm(self.head_dim, eps=config.rms_norm_eps)
modeling_laguna.py · L363–364
query_states = self.q_norm(query_states).transpose(1, 2)
        key_states = self.k_norm(key_states).transpose(1, 2)
modeling_laguna.py · L384–385
Partial RoPE code only in its codenot stated

Full-attention layers use partial_rotary_factor 0.5 so RoPE is applied to only half of each head dimension.

rope_parameters.full_attention.partial_rotary_factor = 0.5
partial_rotary_factor = config.rope_parameters[layer_type].get("partial_rotary_factor", 1.0)
        head_dim = getattr(config, "head_dim", None) or config.hidden_size // config.num_attention_heads
        dim = int(head_dim * partial_rotary_factor)
modeling_laguna.py · L109–111
YaRN RoPE scaling code only in its codenot stated

The full_attention layer type sets rope_type to yarn with factor/beta parameters, and the rotary embedding module dispatches to the YaRN init function.

rope_parameters.full_attention.rope_type = "yarn"rope_parameters.full_attention.factor = 128.0
self.rope_type[layer_type] = rope_params["rope_type"]
            rope_init_fn: Callable = self.compute_default_rope_parameters
            if self.rope_type[layer_type] != "default":
                rope_init_fn = ROPE_INIT_FUNCTIONS[self.rope_type[layer_type]]
modeling_laguna.py · L81–84
Layers without positional encoding (NoPE) not in its codenot stated

Every layer type (full_attention and sliding_attention) has an entry in rope_parameters with a rope_type, so all softmax-attention layers receive positional encoding.

rope_parameters.sliding_attention.rope_type = "default"
cos, sin = position_embeddings
        query_states, key_states = apply_rotary_pos_emb(query_states, key_states, cos, sin)
modeling_laguna.py · L388–389
Mixture of experts in its code core

LagunaSparseMoeBlock routes tokens to top-10 of 256 experts via a learned sigmoid router, applied in layers marked sparse.

num_experts = 256num_experts_per_tok = 10
_, routing_weights, selected_experts = self.gate(hidden_states)
        hidden_states = self.experts(hidden_states, selected_experts, routing_weights)
modeling_laguna.py · L242–243
Shared expert in its code used

Each sparse MoE block includes a shared_experts MLP with nonzero intermediate size 1024 that every token passes through, added to the routed output.

shared_expert_intermediate_size = 1024
self.shared_experts = LagunaMLP(config, intermediate_size=config.shared_expert_intermediate_size)
modeling_laguna.py · L234
hidden_states = hidden_states + shared_output
modeling_laguna.py · L246
Auxiliary-loss-free load balancing (selection bias) code only in its codenot stated

The router adds e_score_correction_bias only to the scores used for top-k selection, not to the routing weights used to combine expert outputs.

scores_for_selection = routing_scores + self.e_score_correction_bias.to(routing_scores.dtype)
        _, selected_experts = torch.topk(scores_for_selection, self.top_k, dim=-1)
        routing_weights = routing_scores.gather(-1, selected_experts)
modeling_laguna.py · L180–182
Multi-token prediction layers not in its codenot stated

No config key declares extra next-next-token prediction modules and no MTP module exists in the modeling code.

SwiGLU feed-forward code only in its codenot stated

Both the dense LagunaMLP and expert MLPs compute silu(gate_proj(x)) * up_proj(x), and hidden_act is silu.

down_proj = self.down_proj(self.act_fn(self.gate_proj(x)) * self.up_proj(x))
modeling_laguna.py · L153

model architecture 19

token mixer 9

Dense gated attention with full RoPE and full gating evaluated

softmax attention 6

Softplus attention gating core

sliding window attention 3

512-token Sliding Window Attention evaluated
Per-Head Gating evaluated
SWA-1024 evaluated

channel mixer 7

positional encoding 3

Per-layer-type rotary position scales core
Rotary Position Embedding used
Gated attention with partial RoPE evaluated

training objective 2

distillation objective 2

Hidden-state caching for KL distillation used
Quantization-aware distillation used

optimization 13

optimizer 2

Muon core
Moonlight-style learning-rate scaling used

learning-rate schedule 2

Warmup-Stable-Decay used
WSD-specific scaling law used

training precision 2

BF16 mixed-precision training default
FP8-precision reinforcement learning used

training stability 2

Cross-replica model-weight hash consistency checks used
Exponential moving average of checkpoints used

training runtime 5

In-flight recovery system core
GPU-to-GPU weight synchronization over GPUDirect RDMA used
Job-level eviction and reclaim used
Pre-admission hardware stress testing used
Same-node sticky pod respawn used

data curation 29

data sourcing 2

High-recall web-data curation used
Streaming training-data ingestion used

data filtering 9

Conservative model-based noise filtering core
Continuous contribution-score ranking core
Multi-dimensional document quality scoring core
Quality-bucket sampling core
Dense annotation of ambiguous low-quality data used
PCA-based dimension selection for decorrelated quality signals used
Propella used
Two-axis document-quality labeling by noise and information used
One-task-per-repository diversity filtering optional

deduplication 1

Snapshot-level fuzzy deduplication used

synthetic data 10

Dynamic multi-agent synthetic-data generation loop core
Modular synthetic-data pipeline composition core
Matching synthesis-pipeline complexity to teacher capability used
Metadata-conditioned synthetic generation used
Multi-stage synthetic-data generation cascade used
Synthetic data augmentation of organic data used
Synthetic system-message augmentation used
Targeted synthetic data rephrasing used
Training-task generation from real commit history used
Two-sided test validation for synthetic coding tasks used

data mixture & curriculum 5

AutoMixer core
Blind pairwise bucket-boundary calibration used
KL-regularized data-mixture optimization used
Pass-rate-based RL task sampling used
Prior-constrained Dirichlet mixture exploration used

sequence packing 1

Sequence packing used

tokenization 1

Subtoken averaging used

post-training 27

filed at the root 1

Reinforcement learning for low-pass-rate tasks core

supervised fine-tuning 4

SFT bootstrapping with synthetic data used
Evaluation-based early stopping used
Frozen-network warmup for new special tokens used
Instruction-following fine-tuning used

reinforcement learning algorithm 5

CISPO with Length-Weighted Leave-One-Out Group-Relative Advantages core
Agentic RL Task Mix used
Per-Token Tool-Error Step Penalty used
Reinforcement Learning from Code Execution Feedback used
Moonlight Scaling optional

reward modelling 4

Binary task verifier core
Binary terminal-verifier reward core
Deterministic chain of checkers core
Per-token tool-error reward shaping used

rollout & RL infrastructure 8

Token-in-token-out (TITO) used
Blocking in-flight rollout steps on weight updates used
Capping trajectory staleness used
Closed-loop multi-turn rollout used
Exact rollout-to-production chat-template alignment assertion used
Long-horizon rollout budgets used
Multi-harness rollouts used
RL sandboxing with selective network blocking and artifact caching used

agentic post-training 4

Production-harness-matched reinforcement learning core
Reinforcement learning in an agent harness core
Multi-harness trajectory training with native behavior preservation used
Seed-based terminal task generation used

mid-training & continual pretraining 1

Continual pretraining for long-context extension used

inference & serving 24

decoding strategy 1

DFlash optional

reasoning control 7

Interleaved thinking between tool calls core
Configurable reasoning effort default
Maximum thinking effort default
Thinking mode with automatic test-time compute budget default
Cross-turn persistent reasoning history used
Thinking modes (off and max) used
User-configurable thinking-effort control not used

KV cache management 1

Inference-side KV-cache reset on weight synchronization used

inference quantization 8

FP8 KV-cache quantization used
AWQ INT4 weight quantization (W4A16) used
FP8 W8A8 quantization with dynamic activation scaling used
Mixed-precision INT4/INT8 layer-wise quantization used
Post-quantization of MoE model layers used
SpinQuant R1 rotation used
TRT-LLM inference optimization to NVFP4 on Blackwell used
In-flight block-wise FP8 weight quantization not used

inference scheduling 1

Envoy-based proxy with custom orchestrator used

context management 1

Preserved thinking history mode used

agentic scaffolding 5

Current-turn-only reasoning-mode detection used
Streaming-delta handling across block boundaries used
XML-tagged tool-call format used
Checkpointing optional
Task-specification prompting mentioned

software implementation 9

filed at the root 1

Shared composable codebase for research and production used

inference engine 1

Atlas inference library used

training framework 2

DAG-based asset lineage tracking core
Experiments as Code core

infrastructure service 5

End-to-end lineage core
HTTP/HTTPS sidecar proxy used
In-house batch scheduler used
Model Factory used
Vendoring external dependencies used

evaluation 12

filed at the root 5

avg@k used
Diverse benchmark validation for quantization used
Four-run mean pass@1 used
Full trajectory release used
Official-release-source baseline restriction used

benchmark 1

Execution-grounded evaluation used

evaluation harness 3

Harbor used
Automatic in-flight checkpoint evaluation scheduling used
Pool used

judge 3

Decompositional instruction-following judge used
Human-calibrated LLM-as-a-Judge used
Reward-hack detection judge used

other 5

filed at the root 5

Automating Repetitive Infrastructure Work core
Agentic Workflows used
Prompt Addendum Prohibiting Direct Online-Solution Use used
Removing Git-History Leaks from Benchmark Images used
Unique Run Identifiers used

unfiled 23

Agentic edit-and-validate loop (read, edit, run checks) core
agentic repository installation task used
automated agentic research loop for software optimization used
benchmark-instrumented target code for agent optimization used
Configuration-flag promotion of validated components to production used
Fixing dependency drift in benchmark tasks used
Fixing verifier selections in benchmark tasks used
Ignoring known teardown flakes in evaluation used
In-task retries for flaky external services used
one-change-at-a-time benchmarking with measurable-wins retention used
open-weight model release used
Parabola fit in log10 learning rate space to find optimal learning rate used
Per-cell power-law fits for loss vs tokens used
Pinning dependency versions to avoid drift used
Publication of unedited evaluation trajectories used
sandboxed agentic benchmarking on a Harbor fork with step limit used
training objective for persistence and verification behaviors used
WSD optimal learning rate scaling law used
cross-provider failover routing optional
Multi-precision weight formats (BF16/FP8/INT4/NVFP4) optional
Checking for agent-generated test scripts before committing mentioned
Incremental task decomposition for agent prompts mentioned
Planning with the model for self-contained tasks mentioned

Further reading

Picked by hand, not extracted: where to read more, not evidence for anything on this page.

Sources

The curated document list for this model. "not fetched" means the URL is recorded but its text was not read in the current run.

DocumentKindPublisherStatus
poolside.ai/blog/introducing-laguna-s-2-1official blogPoolsideread
poolside.ai/assets/laguna/laguna-m1-xs2-technical-report.pdftechnical reportPoolsideread
huggingface.co/poolside/Laguna-S-2.1model cardPoolsideread
docs.poolside.ai/release-notes/modelsvendor docsPoolsideread
poolside.ai/modelsvendor docsPoolsideread
venturebeat.com/infrastructure/poolside-drops-laguna-s-2-1-an-open-weig…newsVentureBeatnot fetched
marktechpost.com/2026/07/21/poolside-releases-laguna-s-2-1newsMarkTechPostread
openrouter.ai/poolside/laguna-s-2.1vendor docsOpenRouterread
finance.yahoo.com/technology/ai/articles/poolside-releases-laguna-2-1-1…newsYahoo Financeread