single mention · data curation
Long-context training-data construction
Long-context training-data construction involves curating datasets with long-context documents to enable reasoning over large documents.1 This is achieved by length-bucketing the training pool and blending it with OCR'd PDFs, which contain documents up to 256k tokens.2
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
we curated datasets that contained long-context documents based on publicly available data (e.g., from German legislative documents) with corresponding question/answering prompts
— Aleph-Alpha/Kolibri-1 · Hugging FaceLong-context candidates were built by length-bucketing the mid-training pool and blending it with OCR'd PDFs
— Aleph-Alpha/Kolibri-1 · Hugging Face
Also called Long-context.
Filed under data curation › data sourcing