Open Models Almanac
Methodsmodel architecturemultimodal architecture

single mention · model architecture

PixelUnshuffle visual-token downsampling

PixelUnshuffle is an operation applied in the connector that lowers the number of visual tokens.1 The step reduces the number of image tokens by a factor of four, yielding substantial savings in memory and compute.2

Trade-off It yields substantial savings in memory and compute with only a slight degradation in downstream multimodal performance.3

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. This connector first applies a PixelUnshuffle (Shi et al., 2016) operation that lowers the number of visual tokens — LFM2 Technical Report
  2. The PixelUnshuffle step reduces the number of image tokens by a factor of four, yielding substantial savings in memory and compute — LFM2 Technical Report
  3. yielding substantial savings in memory and compute with only a slight degradation in downstream multimodal performance in our experiments — LFM2 Technical Report

Also called PixelUnshuffle.

Filed under model architecture › multimodal architecture