single mention · model architecture
PixelUnshuffle visual-token downsampling
PixelUnshuffle is an operation applied in the connector that lowers the number of visual tokens.1 The step reduces the number of image tokens by a factor of four, yielding substantial savings in memory and compute.2
Trade-off It yields substantial savings in memory and compute with only a slight degradation in downstream multimodal performance.3
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
This connector first applies a PixelUnshuffle (Shi et al., 2016) operation that lowers the number of visual tokens
— LFM2 Technical ReportThe PixelUnshuffle step reduces the number of image tokens by a factor of four, yielding substantial savings in memory and compute
— LFM2 Technical Reportyielding substantial savings in memory and compute with only a slight degradation in downstream multimodal performance in our experiments
— LFM2 Technical Report
Also called PixelUnshuffle.
Filed under model architecture › multimodal architecture