SourcesLiquid AI
publisher · 5 sources
Liquid AI
5 sources read from this publisher, newest first, with the methods each one describes.
Introducing LFM3: Every Modality for Specialized Intelligence | Blog
first party release · official blog · 2026-10-27 · 34 methods
Multi-Teacher On-Policy DistillationSupervised fine-tuningLong-context extension via continued pretrainingSliding Window AttentionAgentic reinforcement learningGlobal attentionMid-trainingMixed-domain reinforcement learningCurated higher-quality mid-training dataGated short convolutionHardware-in-the-loop architecture searchLength penaltyPartial rotary position embeddingsEnd-to-end fine-tuningHead-wise gated attentionHybrid architectureRobot control through tool orchestrationSandwich normalization128K-token vocabularyDual proprioception conditioningFine-tuned skill monitor for robot recoveryFull RoPE in sliding-window attention layersKernel-size-7 convolutionLarge-scale pre-training on 22 trillion tokensLocal fine-tuning for specializationParallel action predictionRank-32 Low-Rank AdaptationRepurposed navigation vocabulary tokensResidual-MLP action head for single-pass prediction of eight-step action chunksResidual-quantized trajectory primitivesSFT and GRPO for domain specializationTask-specific fine-tuning for wind-turbine blade defect detectionTri-hybrid layer allocation with 13 convolution, 5 sliding-window attention, and 6 global-attention layersVision-and-language navigation
LiquidAI/LFM3-1.7B-Base · Hugging Face
software documentation · model card · 2026-10-27 · 9 methods
Build specialized multimodal models with LFM3
first party release · official blog · 2026-10-27 · 19 methods
Sliding Window AttentionGlobal attentionCurated higher-quality mid-training dataGated short convolutionHardware-in-the-loop architecture searchPartial rotary position embeddingsCausal audio encoder with sliding-window attention and short convolutionsHead-wise gated attentionRobot control through tool orchestrationSandwich normalizationSingle-GPU fine-tuningConvolution-attention vision encoderJoint training across multiple robot embodimentsLiquidCodec audio tokenizer with streaming decodingPairwise sampled-frame embeddingSingle-pass continuous action chunk predictionTri-hybrid architecture combining short convolutions and attentionTri-hybrid layer mix of gated short convolutions, sliding-window attention, and global attentionVisual route following with target and waypoint prediction
LiquidAI/LFM3-1.7B · Hugging Face
software documentation · model card · 2026-10-27 · 21 methods
Multi-Teacher On-Policy DistillationSupervised fine-tuningLong-context extension via continued pretrainingAgentic reinforcement learningMid-trainingMixed-domain reinforcement learningLength penaltyReasoning enabled by defaultTask-specific sampling-parameter presetsAgentic post-trainingDeferred tool loadingSingle-GPU fine-tuningTri-hybrid model architectureApplication-managed tool executionChatMLHarness-managed search for deferred toolsMain pre-training with an expanded domain mixRemove prior-turn reasoning unless preservedRetrieval or searchTemplate-based tool formattingTwo-stage pre-training
LFM2 Technical Report
technical report · 2025-11-28 · 108 methods
Mixture of expertsSupervised fine-tuningGrouped-query attentionGated short convolutionGreedy decodingHardware-in-the-loop architecture searchChain-of-thought few-shot promptingConditional Top-K-only temperature scalingDecoupled Top-K knowledge distillationDifficulty-ordered SFT curriculum based on ensemble solve ratesParameter-space model mergingDAREDELLAExact text deduplicationJudge-LLM ensemble scoringLength-normalized direct preference alignmentModel soupTask arithmeticThree-stage post-training pipelineTIES-MergingASR-based connector alignmentAutomated and LLM-based safety filteringAutomatic prompt truncationBenchmark decontamination by n-gram and semantic matchingChain-rule decomposition of Top-K KLCMinHash locality-sensitive hashingConnector pre-training with frozen encodersContinuous-input, discrete-output audio modelingContrastive Learning from AI Revisions (CLAIR)Conversion of text SFT datasets into synthetic audio conversationsCross-category integration of multilingual samples in SFT dataCross-encoder-to-ColBERT knowledge distillationCustom internal evaluation harnessDARE combined with standard task arithmeticDARE combined with TIES-style consensus mergingData annealing as an alternative to connector pre-trainingDataset recaptioning for multimodal trainingDense-first-layer, sparse-MoE-later-layer placementDiscrete Residual Vector Quantization audio generationDisjoint majority-sign merge