Open Models Almanac
Methodspost-trainingsupervised fine-tuning

single mention · post-training

Long-context post-training with document question answering

Long-context post-training with document question answering extends a model to reason over large documents by fine-tuning it on long documents paired with question/answering prompts.1 It is needed because most datasets focus on short contexts, so long-context data must be curated, and the training is continued during reinforcement learning.2

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. we curated datasets that contained long-context documents based on publicly available data (e.g., from German legislative documents) with corresponding question/answering prompts — Aleph-Alpha/Kolibri-1 · Hugging Face
  2. Since most datasets usually focus on short-context, we curated datasets that contained long-context documents — Aleph-Alpha/Kolibri-1 · Hugging Face

Also called Long Context.

Filed under post-training › supervised fine-tuning