Open Models Almanac
Methodsdata curationdata filtering

single mention · data curation

Independent annotator quality checks

Independent annotator quality checks are a QA process in which each labeled instance is quality-checked by a different annotator than the one who labeled it.1 A subset of labeled instances, selected based on annotator performance, additionally undergoes auditing by a dedicated team of experienced auditors following the same QA guidelines.2

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. Each labeled instance first undergoes a quality check performed by a different annotator — Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail
  2. of labeled instances are selected, based on the performance of the assigned annotators, for an additional auditing process conducted by a dedicated team of experienced auditors — Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail

Also called Quality Assurance (QA).

Filed under data curation › data filtering