Open Models Almanac
Methods

single mention · not yet filed

VQ-VAE depth-map quantization into discrete depth-token codes

A VQ-VAE encodes depth maps into discrete depth-token codes by learning a latent embedding and mapping it to a learned codebook.1 It is trained with a reconstruction objective on depth maps, so each image's depth map can be expressed as a tokenized string of tokens.2

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. we encode each observation image with the VQVAE to get their latent embeddings. We then represent the latent embedding with a learned codebook — MolmoAct Action Reasoning Models that can Reason in Space
  2. The VQVAE is trained with a standard reconstruction objective to minimize reconstruction loss between input RGB images and their corresponding depth maps — MolmoAct Action Reasoning Models that can Reason in Space

Also called VQVAE.