implementation detail · filed under evaluation
Mandatory agentic judge protocol
An agentic judge must read the output, create a rubric, score candidates against it, and record scores in a scorepad.
Also called mandatory protocol for agentic judge.
- source
- 1
- model
- 1
- lab adopt it
- 1
- strongest
- used
How sources treat it
One count per evidence span, weakest treatment to strongest.
used 1
Documented in
Evidence
1 span quoted from the sources, strongest treatment first.
the agentic judge is required to follow a mandatory protocol: (1) read the outcome, product, or text output; (2) generate a rubric; (3) score each candidate against the rubric; and (4) record the rubric-assigned scores in a scorepad.
usedunclearin Kimi K3Moonshot AI
Filed alongside
Other methods under evaluation :: judge.
Agent-as-a-JudgeIndependent quality-inspection agentAgent-as-a-VerifierDecompositional instruction-following judgeDeterministic tool-call verifierGPT-5.5 (medium) judge modelHuman-calibrated LLM-as-a-JudgeLLM autograding with expert validationLLM-as-a-JudgeLLM-based anti-cheating judgmentLLM-based correctness judgingLLM-based external API usage inspectionOfficial task verifier scoringReward-hack detection judgeRule-based anti-cheat checks