Open Models Almanac
Methodsmodel architecturemultimodal architecture

general family · filed under model architecture

Vision Transformer (ViT)

A Vision Transformer (ViT) cuts an image into fixed-size square patches, maps each patch to a vector with one linear layer, adds position information and runs a standard transformer encoder over the patch sequence. Its cost grows with the number of patches, which is the image area divided by the square of the patch size.

Written for the explainer Getting images and video into a decoder: encoders, projectors, and native fusion.

Also called Vision Transformer, ViT vision encoder.

sources
2
models
2
labs adopt it
2
strongest
used

Documented in

Evidence

2 quotes from 2 sources, one card per model, each opening with its strongest.

How sources treat it
used 2
↑