single mention · evaluation
PaperBench hierarchical rubric evaluation
PaperBench is a benchmark that evaluates AI agents' ability to replicate AI research papers, including understanding contributions, developing a codebase, and executing experiments.1 It grades replication hierarchically: rubrics decompose each task into smaller sub-tasks with clear grading criteria, yielding many individually gradable tasks.2
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
PaperBench [35] evaluates the ability of AI agents to replicate state-of-the-art AI research
— gpt-oss-120b & gpt-oss-20b Model Cardwe develop rubrics that hierarchically decompose each replication task into smaller sub-tasks with clear grading criteria
— gpt-oss-120b & gpt-oss-20b Model Card
Also called PaperBench.
Filed under evaluation › benchmark