Open Models Almanac
Methodsmodel architecturepositional encoding

single mention · model architecture

Shared spatial and modality-specific positional embeddings

A positional-encoding scheme for RGB-D ViT inputs that encodes both the 2D spatial location of each token and its modality identity.1 It adds a shared learnable 2D spatial positional embedding used by both RGB and depth tokens to a modality embedding that marks each token as RGB or depth.2

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. RGB-D token sequences require encoding two types of positional information: (1) the 2D spatial location of each token, and (2) the modality identity — Masked Depth Modeling for Spatial Perception
  2. a shared learnable 2D spatial positional embedding for both RGB and depth tokens, capturing their spatial locations in the image plane; and (2) a modality embedding that distinguishes the input source of each token—RGB or depth — Masked Depth Modeling for Spatial Perception

Also called two types of positional embeddings.

Filed under model architecture › positional encoding