Open Models Almanac
Methodsevaluationjudge

single mention · evaluation

LLM-as-a-Verifier

LLM-as-a-Verifier uses an LLM as a verifier to select one candidate trajectory from a pool, without extra training or a separately trained verifier.1 It grades candidates on multiple criteria with a graded scale, and combines pairwise comparisons via a Bradley–Terry preference probability in a tournament.2

Trade-off It requires no additional parameter updates or separately trained verifier.3

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. The same checkpoint then serves as a verifier, selecting one candidate using LLM-as-a-Verifier (LAV — NVIDIA Nemotron 3.5 Super and Lightning Technical Report
  2. converted into a preference probability using the Bradley–Terry model — NVIDIA Nemotron 3.5 Super and Lightning Technical Report
  3. This requires neither additional parameter updates nor a separately trained verifier — NVIDIA Nemotron 3.5 Super and Lightning Technical Report

Also called LLM-as-a-Verifier candidate selection, LLM-as-a-Verifier (LAV).

Filed under evaluation › judge