Open Models Almanac
ModelsLFM3-1.7B

Liquid AI

LFM3-1.7B

What LFM3-1.7B’s own documents say it is built from — every method with the sentence that describes it.

Parameters
1.73B dense
Context
256K
Attention
conv hybrid13 gated short-conv (k=7) : 5 sliding-window (256) : 6 global GQA layers
Experts
dense
Input
text
Licence
LFM Open License v1.0
Released
2026-10-27

Against the consensus recipe

Its documents state 4 of the 18 methods the field agrees on.

The recipe from the overview: the methods the most labs adopt, per stage. A missing one is something its documents do not say, not something the model lacks.
  1. Data curation

  2. Training objective

  3. Model architecture

  4. Optimization

  5. Post-training

  6. Inference & serving

Checked in its code

Its code shows 7 of the 19 architecture features checked — 4 of them are not stated in its documents.

Sources say what a lab chose to describe; the modeling code says what the checkpoint runs. Each feature below was put to a code verifier, and only the config values and source lines it cited that were found in the files are shown, linked to the line.

Read from Hugging Face Transformers (modeling_lfm3.py, configuration_lfm3.py) and config.json at revision 0dc4268b56, by anthropic/claude-sonnet-5 on 2026-11-07.

FeatureCodeIts documentsVerdict
Grouped-query attention in its codeadopted

16 query heads share 4 KV heads via repeat_kv in every attention layer (full and sliding).

show evidencenum_attention_heads = 16num_key_value_heads = 4
self.num_key_value_groups = config.num_attention_heads // config.num_key_value_heads
modeling_lfm3.py · L218
Multi-head latent attention not in its codenot stated

No low-rank KV latent projection exists; attention uses standard per-head q/k/v projections sized by num_key_value_heads.

Sliding-window attention in its codeadopted

Layers typed sliding_attention set self.sliding_window = config.sliding_window = 256 and a sliding window mask is built for them.

show evidencesliding_window = 256layer_types = ["conv", "conv", "full_attention", "conv", "sliding_attention", "conv", "full…
self.sliding_window = config.sliding_window if self.layer_type == "sliding_attention" else None
modeling_lfm3.py · L237
Interleaved sliding-window and global attention code only in its codenot stated

layer_types interleaves full_attention and sliding_attention layers (plus conv layers) in a fixed pattern, with separate mask builders per…

show evidence

layer_types interleaves full_attention and sliding_attention layers (plus conv layers) in a fixed pattern, with separate mask builders per type.

layer_types = ["conv", "conv", "full_attention", "conv", "sliding_attention", "conv", "full…
"full_attention": lambda: create_causal_mask(**mask_kwargs),
                "sliding_attention": lambda: create_sliding_window_causal_mask(**mask_kwargs),
                "conv": lambda: create_recurrent_attention_mask(**mask_kwargs),
modeling_lfm3.py · L553–555
Indexer-selected sparse attention (DeepSeek Sparse Attention) not in its codenot stated

No indexer scoring module or top-k token selection mechanism exists in the provided code.

Linear-attention or state-space layers alongside full attention not in its codenot stated

Non-attention layers use a short causal convolution (Lfm3ShortConv), not a linear-attention or SSM recurrent mixer, alongside full/sliding…

show evidence

Non-attention layers use a short causal convolution (Lfm3ShortConv), not a linear-attention or SSM recurrent mixer, alongside full/sliding softmax attention.

layer_types = ["conv", "conv", "full_attention", "conv", "sliding_attention", "conv", "full…
class Lfm3ShortConv(nn.Module):
modeling_lfm3.py · L341
Gated DeltaNet layers not in its codenot stated

No delta-rule recurrent state update with a decay gate is implemented; the non-attention layers are plain causal convolutions.

Mamba-2 layers not in its codenot stated

No Mamba-2 SSD selective state-space block is implemented; non-attention layers are short causal convolutions (Lfm3ShortConv), not SSM.

Learnable attention sink not in its codenot stated

No learned per-head sink logit is added to the attention softmax denominator anywhere in the code.

Gated attention output code only in its codenot stated

use_attention_gate is true, so g_proj produces a per-head sigmoid gate that multiplies the attention output before the output projection.

show evidencemodel_type = "lfm3"
if self.g_proj is not None:
            attn_output = attn_output * torch.sigmoid(self.g_proj(hidden_states)).unsqueeze(-1)
modeling_lfm3.py · L280–281
QK normalization code only in its codenot stated

Queries and keys are each passed through an RMSNorm over the head dimension before RoPE and the attention dot product.

show evidence
query_states = self.q_norm(self.q_proj(hidden_states).view(hidden_shape)).transpose(1, 2)
        key_states = self.k_norm(self.k_proj(hidden_states).view(hidden_shape)).transpose(1, 2)
modeling_lfm3.py · L255–256
Partial RoPE in its codeadopted

full_attention layers use partial_rotary_factor 0.5, applying RoPE to only half of the head dimension.

show evidencerope_parameters.full_attention.partial_rotary_factor = 0.5
YaRN RoPE scaling not in its codenot stated

rope_parameters entries specify rope_type "default" for both layer types, not YaRN, and no YaRN scaling code path is exercised.

show evidencerope_parameters.full_attention.rope_type = "default"rope_parameters.sliding_attention.rope_type = "default"
Layers without positional encoding (NoPE) not in its codenot stated

All attention layer types (full_attention, sliding_attention) have an entry in rope_parameters and receive rotary embeddings; only…

show evidence

All attention layer types (full_attention, sliding_attention) have an entry in rope_parameters and receive rotary embeddings; only non-attention conv layers lack RoPE, which does not count.

rope_parameters = {"full_attention": {"partial_rotary_factor": 0.5, "rope_theta": 1000000.0, "r…
self.layer_types = [layer_type for layer_type in config._rope_type_labels if layer_type in config.layer_types]
modeling_lfm3.py · L72
Mixture of experts documents disagree not in its codeadopted

No router or expert MLP modules exist; Lfm3MLP is a single dense SwiGLU feed-forward applied in every decoder layer.

show evidence
class Lfm3MLP(nn.Module):
modeling_lfm3.py · L125
Shared expert not in its codenot stated

There is no MoE mechanism at all, so there is no shared-expert path either.

Auxiliary-loss-free load balancing (selection bias) not in its codenot stated

No router or expert-selection bias exists since the model has no mixture-of-experts layers.

Multi-token prediction layers not in its codenot stated

No extra next-next-token prediction modules are declared in config or built in the modeling code.

SwiGLU feed-forward code only in its codenot stated

Lfm3MLP computes SiLU(w1(x)) * w3(x) then projects with w2, a SwiGLU feed-forward used by every layer.

show evidence
def forward(self, x):
        return self.w2(F.silu(self.w1(x)) * self.w3(x))
modeling_lfm3.py · L134–135

filed at the root 1

Hybrid architecture core

“Built on a new hybrid architecture”

token mixer 7

Gated short convolution core

“uses gated short convolutions for most layers and a small number of GQA blocks for global context”

Input-dependent multiplicative gating around a depthwise short convolution core

“each gated short convolution block applies input-dependent multiplicative gating around a depthwise short convolution”

Kernel-size-7 convolution core

“The convolutions also use a kernel size of 7, up from 3 in LFM2.”

softmax attention 4

global attention 1

Global attention core

“LFM3 combines three operators: gated short convolutions, sliding-window attention (SWA), and global attention.”

sliding window attention 1

Sliding Window Attention core

“sliding-window attention (SWA) gave the best quality–speed tradeoff.”

grouped-query attention 2

Grouped-query attention core

“interleaved with a small minority of GQA blocks”

Sparse use of grouped-query attention blocks for global context core

“a small number of GQA blocks for global context”

channel mixer 3

mixture of experts 3

Mixture of experts core

“replace dense MLPs with sparse MoE MLPs in most layers”

Dense-first-layer, sparse-MoE-later-layer placement used

“the first two layers remain dense while all subsequent layers include an MoE block”

expert routing 1

Top-k routing with normalized sigmoid and adaptive load-balancing bias core

“selects the Top-k experts per token with a normalized sigmoid router and adaptive routing biases for load balancing”

positional encoding 2

Partial rotary position embeddings core

“Global attention layers rotate only the highest-frequency half of each head and leave the low-frequency half unrotated”

Full RoPE in sliding-window attention layers core

“SWA layers keep full RoPE.”

normalization & residual 2

Head-wise gated attention core

“A sigmoid gate scales each head’s output, so a head can switch off when it has nothing to add.”

Sandwich normalization core

“each block normalizes both its input and its output to keep activations well-scaled”

prediction head 1

Residual-MLP action head for single-pass prediction of eight-step action chunks core

“a small residual MLP maps the hidden states of eight action-query tokens to an eight-step chunk of continuous robot actions in a single…”

multimodal architecture 20

Causal audio encoder with sliding-window attention and short convolutions core

“is fully causal, so the model can prefill while you speak”

Convolution-attention vision encoder core

“it combines convolution and attention and uses 3x less compute than the SigLIP2 encoder”

Discrete Residual Vector Quantization audio generation core

“audio generation is implemented through a discrete Residual Vector Quantization (RVQ) code path”

Dual proprioception conditioning core

“Proprioception is provided both as discretized text and through an embodiment-specific learned projection”

Eight-codebook Mimi RVQ codes core

“Specifically, the model predicts 8-codebook Mimi RVQ codes”

Pairwise sampled-frame embedding core

“it embeds each pair of sampled frames into one temporal slice, halving the temporal sequence length”

Three-stage multimodal training protocol core

“We train LFM2-VL in three stages, mirroring the language-only pipeline and introducing multimodal alignment at appropriate points.”

Vision-language extension with feature connector core

“All LFM2-VL variants retain the decoder-only LFM2 backbone and attach a SigLIP2 image encoder through a small connector that maps visual…”

ASR-based connector alignment used

“In the alignment phase, we use ASR data to train the MLP connector, while keeping the LFM2 backbone and encoder frozen.”

Connector pre-training with frozen encoders used

“The first stage focuses on aligning the visual and textual embedding spaces while keeping the image encoder and language backbone frozen”

Dynamic image tiling used

“For higher-resolution inputs, we switch to dynamic tiling.”

Fixed 6:12 text-to-audio token interleaving used

“In our implementation, we generate 6 text tokens followed by 12 audio tokens and repeat until all text tokens are exhausted.”

Parallel action prediction used

“we adopt parallel action prediction inspired by OpenVLA-OFT and StarVLA-α”

PixelUnshuffle visual-token downsampling used

“This connector first applies a PixelUnshuffle (Shi et al., 2016) operation that lowers the number of visual tokens”

Repurposed navigation vocabulary tokens used

“We only repurpose 2,068 vocabulary tokens; the architecture is otherwise unchanged.”

Residual-quantized trajectory primitives used

“Each decision consists of five tokens: two spatial/control tokens and three hierarchically quantized codes encoding a ten-waypoint path”

SigLIP2 vision encoder with dynamic tiling and token reduction used

“adds a SigLIP2-based vision encoder with dynamic tiling and token reduction”

Single-pass continuous action chunk prediction used

“The manipulation VLA maps images, language, and robot state to eight-step chunks of continuous actions in a single forward pass”

Visual route following with target and waypoint prediction used

“The navigator follows route descriptions using recent RGB observations, predicting spatial targets and a waypoint trajectory in just five…”

Global thumbnail alongside tiled image inputs optional

“We optionally append a single thumbnail frame, processed through the same single-frame pipeline as above, to provide a coarse global…”

1 more mentioned or evaluated
Vision-and-language navigation evaluated

multimodal architecture “LFM3-VL-2B reads a route description and a short history of RGB frames to predict navigation targets and residual-quantized trajectory…”

filed at the root 1

GAN training used

“The detokenizer is trained separately as a GAN”

language modelling objective 1

Large-scale pre-training on 22 trillion tokens used

“LFM3-1.7B-Base was pre-trained on 22T tokens.”

distillation objective 6

Chain-rule decomposition of Top-K KL core

“decomposing the KL via the chain rule into (i) a binary term that matches the total probability mass assigned to the Top-K set, and (ii) a…”

Conditional Top-K-only temperature scaling core

“We apply temperature only to the conditional Top-K distributions inside”

Decoupled Top-K knowledge distillation core

“A tempered, decoupled Top-K knowledge distillation objective reduces storage and bandwidth when distilling from a larger teacher”

Cross-encoder-to-ColBERT knowledge distillation used

“We train LFM2-ColBERT-350M using knowledge distillation through the PyLate framework (Chaffin and Sourty, 2024), leveraging pre-computed…”

Mean squared error distillation of relevance scores used

“We employ the distillation loss implemented in PyLate, which minimizes the mean squared error between student and teacher relevance scores.”

Min-max normalization of teacher relevance scores used

“The PyLate framework applies min-max normalization to teacher scores to ensure numerical stability and consistent gradient magnitudes…”

auxiliary loss 1

Weighted averaging of audio codebook losses used

“We combine the 8 audio codebook losses into a single via a weighted average.”

1 more mentioned or evaluated
Teacher-weighted reverse-KL surrogate for Top-K distillation evaluated

distillation objective “we also consider a teacher-weighted surrogate”

filed at the root 1

Two-stage pre-training core

“Pre-training: two stages build LFM3-1.7B-Base on 22T tokens in total.”

training stability 1

Training-only embedding input dropout used

“we apply 10% input dropout on embeddings during training only, while avoiding internal dropout layers to preserve inference latency”

training parallelism 1

ZeRO-2 with strided activation checkpointing used

“The distributed training infrastructure leverages ZeRO-2 sharding with strided activation checkpointing across 8 H100-80GB GPUs per…”

training runtime 1

Linear checkpoint merging in weight space used

“the highest-performing checkpoints are combined with a simple linear merge in weight space”

3 more mentioned or evaluated
Magnitude-aware parameter dropout sampling evaluated

“For each layer, parameters are ranked by absolute magnitude, and dropout probabilities are assigned inversely proportional to rank”

Magnitude-based task-vector sparsification evaluated

“magnitude-based sparsification retains only the top of parameters by absolute value in each task vector”

Majority-sign election evaluated

“sign election resolves conflicts by computing the majority sign for each parameter position across all task vectors”

data sourcing 1

Multilingual vision-language training through translated data used

“across training stages we integrate vision-language data translated into several target languages”

data filtering 6

Multi-stage SFT quality filtering and decontamination core

“We implement a comprehensive multi-stage filtering process to ensure dataset quality and prevent evaluation contamination.”

Automated and LLM-based safety filtering used

“Some large-scale image datasets used at this stage are filtered for safe-for-work content using automated filters and LLM-based screening”

Benchmark decontamination by n-gram and semantic matching used

“we perform decontamination through exact n-gram matching against evaluation benchmarks, with additional semantic similarity filtering to…”

Human screening of candidate SFT datasets used

“we perform human evaluation on a subset of each candidate dataset”

Rule-based and LLM-judged audio data filtering used

“Filtering combines rule-based validation (e.g., duration constraints, transcription alignment, and pattern-based filtering) with…”

Rule-based stylistic filtering of SFT data used

“Stylistic filtering uses rule-based methods to eliminate samples with overused phrases, clichés, and undesirable patterns.”

deduplication 3

Exact text deduplication used

“We perform exact deduplication through direct matching”

CMinHash locality-sensitive hashing used

“apply near-duplicate detection using CMinHash locality-sensitive hashing to catch similar but non-identical content”

Semantic deduplication used

“Semantic deduplication proved most impactful, using sentence embeddings with a high similarity threshold to remove semantically similar…”

synthetic data 9

Conversion of text SFT datasets into synthetic audio conversations used

“pipelines specialized for turning existing text SFT datasets into audio conversations”

Dataset recaptioning for multimodal training used

“To improve mid-training data quality, we recaption existing open-source image datasets”

On-policy preference-data generation and skill-specific refinement used

“The dataset construction focuses on in-distribution samples and combines on-policy generation with skill-specific refinement through…”

Synthetic multi-turn audio conversation data generation used

“A large focus on synthetic conversation data is employed to generate a high volume of multi-turn audio conversations”

Synthetic paralinguistic and discourse feature generation used

“This involves the synthetic generation of speech cues and conversational behaviors such as intonation patterns, pauses, filler words”

Synthetic VQA generation targeted at visual understanding used

“generate additional VQA samples specifically targeted at visual understanding, including fine-grained reasoning about objects, layouts…”

Theme-conditioned synthetic audio conversation generation used

“pipelines that generate audio conversations following a pre-determined theme using randomly sampled seed specifications”

Visual augmentation for robust document OCR used

“We incorporate additional document OCR data with visual augmentations such as blur, noise, and geometric distortions to make the OCR…”

Voice cloning to diversify synthetic user voices used

“Additionally, we sample user voices via voice-cloning techniques to teach the model a wide distribution of user voice characteristics”

data mixture & curriculum 5

Curated higher-quality mid-training data used

“The final 3.5T tokens are a mid-training stage on curated data with a larger share of agentic traces and tool calls”

Difficulty-ordered SFT curriculum based on ensemble solve rates used

“We implement a data-driven curriculum learning strategy across the entire dataset to optimize the training progression from simpler to…”

Main pre-training with an expanded domain mix used

“the mix adds more scientific papers, repository-level code, math, and a wider range of knowledge sources”

Skill-specific SFT mixture design used

“We develop the final SFT composition through an iterative process targeting individual capabilities in isolation before combining them…”

Text-to-multimodal data annealing schedule used

“We apply a data annealing schedule that starts with 100% text-only data and, over the first 5% of steps, linearly decreases its fraction…”

tokenization 2

ChatML core

“LFM3 uses this ChatML format, generated from the checkpoint's chat template”

128K-token vocabulary used

“LFM3 uses the same 128K vocabulary as LFM2.5-8B-A1B, built to tokenize non-Latin scripts more efficiently.”

1 more mentioned or evaluated
Data annealing as an alternative to connector pre-training evaluated

data mixture & curriculum “Ablations showed that the annealing data schedule removes the need for connector pre-training”

filed at the root 3

Three-phase audio model training core

“Training proceeds in three phases: alignment, mid-training, and post-training.”

Three-stage post-training pipeline core

“A three-stage post-training recipe including supervised fine-tuning, length-normalized direct preference alignment, and parameter-space…”

Full post-training using TRL optional

“through continued pre-training or full post-training via TRL”

supervised fine-tuning 10

Supervised fine-tuning core

“Large general-purpose mixture to provide good capabilities on downstream tasks and teach the chat template.”

End-to-end fine-tuning used

“the whole network is fine-tuned end to end.”

Cross-category integration of multilingual samples in SFT data used

“we systematically integrate multilingual samples across most task categories”

Fine-tuned skill monitor for robot recovery used

“Paired with a fine-tuned d1-2B skill monitor, we can further double recovery success”

Joint multimodal supervised fine-tuning used

“The final stage performs supervised fine-tuning on multimodal instruction-following data”

Rank-32 Low-Rank Adaptation used

“A rank-32 LoRA adapter was trained on about 40,000 samples”

Task-specific fine-tuning for wind-turbine blade defect detection used

“we fine-tuned LFM3-VL-2B to find and classify defects in drone photos of wind turbine blades.”

Single-GPU fine-tuning optional

“adapt the model to your domains, formats, and tools via quick single-GPU fine-tuning”

Frozen-backbone projection-layer adaptation optional

“this can be achieved by continuing to train the projection layer while keeping the LFM2 backbone frozen”

Local fine-tuning for specialization optional

“it can be fine-tuned locally”

reinforcement learning algorithm 2

Mixed-domain reinforcement learning used

“Post-training from this checkpoint (SFT, mixed RL, multi-domain on-policy distillation, and agentic RL) produces LFM3-1.7B.”

SFT and GRPO for domain specialization used

“Training used about 3,000 public photos (WTBs2025) with SFT and GRPO for a few hours on one A100.”

reward modelling 1

Length penalty used

“A length penalty in the reward keeps answers concise.”

preference optimization 3

Length-normalized direct preference alignment core

“we combine a family of length-normalized direct alignment objectives with an offline dataset”

Offline direct preference alignment core

“Enabling efficient preference optimization through direct alignment on offline data including on-policy samples generated from the SFT…”

Contrastive Learning from AI Revisions (CLAIR) used

“For instruction following data, we further refine the chosen responses via Contrastive Learning from AI Revisions (CLAIR).”

policy distillation 1

Multi-Teacher On-Policy Distillation used

“six specialist RL teachers (code, math, tool use, long context, agentic work, and enterprise workflows) are distilled back into a single…”

agentic post-training 2

Agentic reinforcement learning used

“Post-training from this checkpoint (SFT, mixed RL, multi-domain on-policy distillation, and agentic RL) produces LFM3-1.7B.”

Agentic post-training used

“Agentic training: post-trained for agentic harnesses such as Hermes Agent, Pi, and OpenClaw”

mid-training & continual pretraining 6

Long-context extension via continued pretraining used

“This stage also extends the context window to 256K tokens.”

Mid-training used

“The last 3.5T tokens use curated, higher-quality data with a larger share of agentic traces and tool calls.”

Joint text-and-audio mid-training used

“Here, we unfreeze the entire model and train on a large-scale mixture of text and audio datasets.”

Mid-training with lower-quality SFT datasets used

“we move these datasets to the mid-training stage instead of using them in SFT”

Multimodal mid-training used

“After language mid-training, we introduce a multimodal mid-training stage that jointly optimizes the language backbone, pre-trained…”

Continued pretraining optional

“a starting point for continued pre-training and your own post-training”

filed at the root 1

Interleaved text-and-audio generation optional

“In interleaved generation mode, LFM2-Audio generates the assistant response in both text and audio, with a predetermined interleaved…”

decoding strategy 2

Greedy decoding used

“Generation: Greedy decoding, max 8192 tokens.”

Task-specific sampling-parameter presets optional

“General (chat, tool use, agents, coding): temperature=0.1,top_k=50, repetition_penalty=1.1”

reasoning control 1

Reasoning enabled by default default

“Reasoning is enabled by default. You can set enable_thinking=False to disable thinking traces”

context management 3

Remove prior-turn reasoning unless preserved default

“Reasoning from turns before the latest user message is removed unless preserve_thinking=True.”

Automatic prompt truncation used

“Automatically truncates prompts exceeding model’s max context length.”

Inference-time vision-token budget scaling optional

“Reducing the token budget introduces minimal degradation on general vision benchmarks”

agentic scaffolding 5

Robot control through tool orchestration used

“it grounds objects by pointing, measures them with depth tools, invokes motion planners, and checks outcomes in subsequent camera images”

Application-managed tool execution used

“Execute the call in your application. Add the parsed assistant call to history as a tool_calls entry, then append the result as a tool…”

Deferred tool loading used

“In the system prompt, tell the model when to search:”

Template-based tool formatting used

“The template formats tool definitions and results automatically.”

Retrieval or search optional

“Use retrieval or search for knowledge-heavy tasks.”

filed at the root 5

Five-shot in-context evaluation default

“Implementation: 5-shot by default (uses first 5 examples from dev set per subject).”

Chain-of-thought few-shot prompting used

“5-shot by default using Chain-of-Thought (CoT) examples from validation set”

Loose-match accuracy used

“Loose score is used because of low performance with strict scoring with Gemma 3 models.”

Randomly sampled five-shot evaluation used

“5-shot examples randomly sampled from train set.”

Repeated evaluation with answer-order permutations used

“Each question evaluated 10 times with different answer orderings to reduce position bias.”

evaluation harness 3

Custom internal evaluation harness used

“The evaluation scores for the text-only models come from a custom internal harness.”

Priority-ordered answer extraction parser used

“Custom parser with priority order:”

Robust answer parsing in a custom evaluation harness used

“we build a custom evaluation harness with robust parsing logic”

judge 2

Judge-LLM ensemble scoring used

“We then score individual responses via an LLM jury, selecting the highest scored sample as the chosen response and the lowest scored…”

On-policy tie-breaking used

“we favor on-policy responses over instruction / preference samples in case of a tie to mitigate distribution shift”

filed at the root 9

Hardware-in-the-loop architecture search core

“We explicitly search this design space with a hardware-in-the-loop search procedure”

MaxSim core

“Following the ColBERT framework (Khattab and Zaharia, 2020), we compute similarity between a query and document using the MaxSim operator”

Parameter-space model merging core

“Systematic evaluation and combination of candidate models to increase robustness and optimize performance.”

Continuous-input, discrete-output audio modeling used

“separates continuous audio input from discrete audio output”

Hypervolume-improvement ranking used

“The remaining candidates are ranked by hypervolume improvement on the quality–latency–memory Pareto frontier.”

LFM2-style audio detokenizer used

“we instead train a compact, LFM2-style detokenizer optimized for edge deployment”

Multi-objective Pareto optimization used

“We frame architecture selection as a Pareto optimization over three axes”

Multi-period discriminator used

“using both a multi-period discriminator (Kong et al., 2020) and a multi-resolution discriminator (Jang et al., 2021)”

Multi-resolution discriminator used

“using both a multi-period discriminator (Kong et al., 2020) and a multi-resolution discriminator (Jang et al., 2021)”

6 more mentioned or evaluated
DARE evaluated

“The techniques we test (model soup, task arithmetic, TIES-Merging, DARE, and DELLA) are described in detail in Appendix B.”

DELLA evaluated

“The techniques we test (model soup, task arithmetic, TIES-Merging, DARE, and DELLA) are described in detail in Appendix B.”

Disjoint majority-sign merge evaluated

“the disjoint merge averages only those parameters that agree with the elected majority sign”

Model soup evaluated

“The techniques we test (model soup, task arithmetic, TIES-Merging, DARE, and DELLA) are described in detail in Appendix B.”

Task arithmetic evaluated

“The techniques we test (model soup, task arithmetic, TIES-Merging, DARE, and DELLA) are described in detail in Appendix B.”

TIES-Merging evaluated

“The techniques we test (model soup, task arithmetic, TIES-Merging, DARE, and DELLA) are described in detail in Appendix B.”

unfiled 18

Tri-hybrid model architecture core

“based on our new tri-hybrid architecture”

late-interaction retrieval core

“We extend the LFM2 backbone to information retrieval through LFM2-ColBERT-350M, a late interaction retriever optimized for multilingual…”

LiquidCodec audio tokenizer with streaming decoding core

“LiquidCodec, our new audio tokenizer, turns generated tokens into speech at about 87x real time on a single CPU thread on Apple M4 Max and…”

Minimal hybrid of gated short convolution blocks and grouped-query attention core

“a minimal hybrid suffices: gated short convolutions for most layers plus a small minority of grouped-query attention (GQA) layers”

Minimal hybrid sequence-modeling architecture core

“LFM2 follows this general principle of hybridization, but adopts a minimal hybrid tuned for devices”

Multi-stage pre-training pipeline core

“We pre-train LFM2 using a multi-stage pipeline designed for small, on-device models.”

RQ-Transformer for autoregressive RVQ code generation core

“Instead, we use a much smaller RQ-Transformer (Lee et al., 2022), similar to the one used in Moshi (Défossez et al., 2024), as a decoder…”

Tri-hybrid architecture combining short convolutions and attention core

“a new tri-hybrid architecture and training recipe allow LFM3-1.7B to outperform LFM2.5-2.6B”

Tri-hybrid layer allocation with 13 convolution, 5 sliding-window attention, and 6 global-attention layers core

“Of its 24 layers, 13 are convolutions, 5 are SWA with a 256-token window, and 6 are global attention.”

Tri-hybrid layer mix of gated short convolutions, sliding-window attention, and global attention core

“13 gated short convolutions, 5 sliding-window attention (SWA) layers with a 256-token window, and 6 global attention layers”

Harness-managed search for deferred tools used

“Your harness implements and runs the search.”

Joint training across multiple robot embodiments used

“Jointly trained across 15 embodiments”

Machine translation of multilingual vision benchmark sets used

“All multilingual benchmarks were translated from English into Arabic, Chinese, French, German, Italian, Japanese, Korean, Portuguese, and…”

single-frame image preprocessing used

“For images in the single-frame regime, we first resize the input so that the SigLIP2 encoder sees a target range of patch counts.”

state-switching sequential text/audio generation optional

“In sequential generation mode, LFM2-Audio can change its own generation state by generating special control tokens.”

2 more mentioned or evaluated
DARE combined with standard task arithmetic evaluated

“We implement DARE in combination with both standard task arithmetic (DARE-linear)”

DARE combined with TIES-style consensus merging evaluated

“with TIES-style consensus (DARE-TIES)”

Not used 1

omission of explicit spatial grounding supervision not used

“we do not employ explicit grounding supervision such as bounding boxes or point coordinates”

Sources

The curated source list for this model. "not fetched" means the URL is recorded but its text was not read in the current run.

SourceKindPublisherStatus
liquid.ai/blog/lfm3official blogLiquid AIread
huggingface.co/blog/LiquidAI/lfm3official blogLiquid AIread
huggingface.co/LiquidAI/LFM3-1.7Bmodel cardLiquid AIread
huggingface.co/LiquidAI/LFM3-1.7B-Basemodel cardLiquid AIread
arxiv.org/abs/2511.23404technical reportLiquid AIread
↑