Open Models Almanac
Methodsevaluation

single mention · evaluation

Vulnerability discovery evaluation

Vulnerability discovery evaluation tasks a model with identifying genuine bugs in current codebases rather than reproducing known vulnerabilities.1 The model must also demonstrate that the bugs it finds are reproducible, capabilities associated with defensive security research.2

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. This tier tasks the model with identifying genuine bugs in current codebases—rather than reproducing known vulnerabilities — k3 tech report
  2. and demonstrating that they are reproducible. These capabilities are primarily associated with defensive security research — k3 tech report

Also called Vulnerability discovery (Tier 1).

Filed under evaluation