single mention · data curation
Independent annotator quality checks
Independent annotator quality checks are a QA process in which each labeled instance is quality-checked by a different annotator than the one who labeled it.1 A subset of labeled instances, selected based on annotator performance, additionally undergoes auditing by a dedicated team of experienced auditors following the same QA guidelines.2
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
Each labeled instance first undergoes a quality check performed by a different annotator
— Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tailof labeled instances are selected, based on the performance of the assigned annotators, for an additional auditing process conducted by a dedicated team of experienced auditors
— Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail
Also called Quality Assurance (QA).
Filed under data curation › data filtering