Open Models Almanac
Methodsmodel architecturetoken mixer

specific method · filed under model architecture

Gated short convolution

A gated short convolution is a token-mixing block that applies input-dependent multiplicative gating around a depthwise short convolution along the sequence.1 It provides fast local mixing, and once a few attention blocks handle long-range retrieval, it can replace linear-attention or SSM branches.2,3

Trade-off It is inexpensive with good cache behavior, but needs a minority of attention blocks for long-range retrieval.4

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. each gated short convolution block applies input-dependent multiplicative gating around a depthwise short convolution — LFM2 Technical Report
  2. This operator provides fast local mixing with excellent cache behavior on CPUs — LFM2 Technical Report
  3. Once a handful of GQA blocks are available to handle long-range retrieval, the inexpensive gated short convolution alone is sufficient — LFM2 Technical Report
  4. a majority of inexpensive, gated short convolution blocks interleaved with a minority of grouped-query attention (GQA) blocks — LFM2 Technical Report

Also called Gated short-convolution layers, Gated short convolutions, Gated short convolutions for local mixing, Input-aware gated short convolution blocks.

sources
3
model
1
lab adopt it
1
strongest
core

Documented in

Liquid AI

Evidence

4 quotes from 3 sources, one card per model, each opening with its strongest.

How sources treat it
core 4

LFM3-1.7B

Liquid AIadopted3 sources

uses gated short convolutions for most layers and a small number of GQA blocks for global context

coreunclearin LFM2Liquid AI
3 more from this model

Gated short convolutions still make up over half the layers.

coremodel architecturein LFM3Liquid AI

most blocks are inexpensive gated short convolution blocks

coremodel architecturein LFM2Liquid AI

13 gated short convolutions

coremodel architecturein LFM3-1.7B-BaseLiquid AI
↑