Open Models Almanac
Methodsevaluationhuman & real-world evaluation

specific method · filed under evaluation

Blind expert evaluation of engineering tasks

Uses a blind evaluation by experts on engineering tasks; the evidence also reports the evaluator and task counts.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See human & real-world evaluation for the methods filed beside it.

Also called Blind side-by-side evaluation, Blind side-by-side human evaluation.

sources
3
model
1
lab adopt it
1
strongest
used

Documented in

Tencent (Hunyuan)

Evidence

3 quotes from 3 sources, one card per model, each opening with its strongest.

How sources treat it
used 3

Hy4-preview

Tencent (Hunyuan)adopted3 sources

we ran a blind side-by-side evaluation: 163 internal experts rated model outputs on 203 engineering tasks.

usedevaluation onlyin Hy4-previewTencent Hy Team
2 more from this model

we ran a blind side-by-side evaluation: 163 internal experts rated model outputs on 203 engineering tasks

usedevaluation onlyin Hy4-previewTencent Hy Team

In an internal blind evaluation involving 163 experts and 203 engineering tasks, Hy4 preview achieved an average score of 2.99 out of 4

usedevaluation onlyin Hy4-previewTencent
↑