single mention · post-training
Long-context post-training with document question answering
Long-context post-training with document question answering extends a model to reason over large documents by fine-tuning it on long documents paired with question/answering prompts.1 It is needed because most datasets focus on short contexts, so long-context data must be curated, and the training is continued during reinforcement learning.2
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
we curated datasets that contained long-context documents based on publicly available data (e.g., from German legislative documents) with corresponding question/answering prompts
— Aleph-Alpha/Kolibri-1 · Hugging FaceSince most datasets usually focus on short-context, we curated datasets that contained long-context documents
— Aleph-Alpha/Kolibri-1 · Hugging Face
Also called Long Context.
Filed under post-training › supervised fine-tuning