single mention · evaluation
Vulnerability discovery evaluation
Vulnerability discovery evaluation tasks a model with identifying genuine bugs in current codebases rather than reproducing known vulnerabilities.1 The model must also demonstrate that the bugs it finds are reproducible, capabilities associated with defensive security research.2
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
This tier tasks the model with identifying genuine bugs in current codebases—rather than reproducing known vulnerabilities
— k3 tech reportand demonstrating that they are reproducible. These capabilities are primarily associated with defensive security research
— k3 tech report
Also called Vulnerability discovery (Tier 1).
Filed under evaluation