specific method · filed under evaluation
Agent-as-a-Judge
An autonomous agent acts as a judge to assess generated projects, with one described setup deploying and building projects before closed-loop inspection.
Also called Agent-as-a-Judge evaluation method.
- source
- 1
- model
- 1
- lab adopt it
- 1
- strongest
- used
How sources treat it
One count per evidence span, weakest treatment to strongest.
Documented in
Evidence
2 spans quoted from the sources, strongest treatment first.
We conduct cross-validation between the Agent-as-a-Judge framework and human experts
We therefore introduce Agent-as-a-Judge: each generated project is deployed in a Docker container and built to verify static correctness. Successfully built instances are then handed to an autonomous Judge Agent (Claude Code with Claude Sonnet 4.5, equipped with Playwright MCP tool) that operates in closed-loop cycles
Filed alongside
Other methods under evaluation :: judge.