single mention · evaluation
SWE-bench Verified evaluation
SWE-bench Verified is a benchmark that evaluates AI models' ability to solve real-world software issues.1 It uses a human-validated subset of SWE-bench that fixes grading errors, under-specified problem statements, and overly specific unit tests, so models are graded accurately.2
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
evaluates AI models’ ability to solve real-world software issues
— gpt-oss-120b & gpt-oss-20b Model Cardthe human-validated subset of SWE-bench that more reliably evaluates
— gpt-oss-120b & gpt-oss-20b Model Card
Also called SWE-bench Verified.
Filed under evaluation › benchmark