Open Models Almanac
Methodsoptimization

single mention · optimization

Unfreezing tokenizer embeddings and language-model head

Unfreezing the tokenizer embedding and language-model head makes these normally frozen layers trainable during fine-tuning.1 It is done so the model can learn to understand and output new tokens, such as depth perception tokens.2

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. unfreezing the tokenizer embedding and lm head — MolmoAct Action Reasoning Models that can Reason in Space
  2. To enable the model to learn to understand and output depth perception tokens, we follow the training scheme of LLaVA-AURORA (Bigverdi et al., 2025) by unfreezing the tokenizer embedding and lm head — MolmoAct Action Reasoning Models that can Reason in Space

Filed under optimization