Open Models Almanac
Methodsevaluationhuman & real-world evaluation

methods · evaluation

human & real-world evaluation

33 methods filed at this node or below it, from the sources of 19 models.

In this branch 33

Everything filed at this node or below it, with one collapsible heading per child node.

filed here 33

Blind expert evaluation of engineering tasks used · 3 sources · 3 quotes
Multi-turn open-ended external red-teaming used · 3 sources · 3 quotes
31 single mentions
Held-out evaluation benchmarks default · 1 source · 1 quote
Automatic success-rate measurement used · 1 source · 1 quote
Blind expert judging used · 1 source · 1 quote
Browser-based testing and iterative repair used · 1 source · 1 quote
Closed-loop simulation used · 1 source · 1 quote
Customer-specific contextualized evaluation used · 1 source · 2 quotes
Cyber range exercises used · 1 source · 1 quote
Dangerous-capability uplift assessment used · 1 source · 1 quote
Human evaluation used · 1 source · 1 quote
Human-verified and LLM-based evaluation used · 1 source · 1 quote
Interleaved policy evaluation used · 1 source · 1 quote
Internal engineering-task evaluation used · 1 source · 2 quotes
On-vehicle road testing used · 1 source · 1 quote
Pairwise comparison for Chinese writing used · 1 source · 1 quote
Pre-release safety evaluation used · 1 source · 1 quote
Real-world feedback refinement used · 1 source · 1 quote
Repeated trials for statistical robustness used · 1 source · 1 quote
Same-prompt comparative DevOps testing used · 1 source · 1 quote
Turn-level blind A/B testing used · 1 source · 1 quote
White-collar enterprise-task evaluation used · 1 source · 1 quote
Blind real-robot testing evaluated · 1 source · 1 quote
A/B testing on real-world tasks mentioned · 1 source · 1 quote
Continuous monitoring mentioned · 1 source · 1 quote

By model

Which of this branch's methods each model's own documents describe, and how strongly. Under each model: its strongest treatment anywhere in the branch.

Modelmethods
Hy4-preview usedBlind expert evaluation of engineering tasks used—
GLM-5.3 usedReal-world codebase testing with security teams used—
DeepSeek-V4-Flash-0731 mentionedA/B testing on real-world tasks mentioned—
Hy3 usedReal-world feedback refinement used—
MiMo-V2.6-Flash usedSame-prompt comparative DevOps testing used—
NVIDIA-Nemotron-3-Ultra-550B-A55B defaultHeld-out evaluation benchmarks defaultAutomated red-teaming for indirect prompt injection used—
Alpamayo 1.5 / 2 Super usedClosed-loop simulation usedHuman-verified and LLM-based evaluation usedOn-vehicle road testing used—
DeepSeek-V4-Pro usedInternal engineering-task evaluation usedPairwise comparison for Chinese writing usedWhite-collar enterprise-task evaluation used—
FLUX 3 Action evaluatedBlind real-robot testing evaluated—
GR00T N1.7 usedAutomatic success-rate measurement usedHuman-observer evaluation of real-robot tasks used—
Gemma 4 31B usedHuman evaluation usedContinuous monitoring mentioned—
Inkling usedDangerous-capability uplift assessment usedMulti-turn open-ended external red-teaming usedPre-release safety evaluation used—
Kimi K3 usedBlind expert judging used—
Kolibri-1 coreCustomer-proxy evaluation without customer data training coreCustomer-specific contextualized evaluation usedInternal checkpoint preview and employee feedback usedTurn-level blind A/B testing used—
LingBot-VLA 2.0 usedRandomized-order sequential model evaluation usedRepeated trials for statistical robustness used—
MiMo-V2.6-Pro usedSame-prompt comparative DevOps testing used—
Mistral-Large-4 usedBrowser-based testing and iterative repair used—
gpt-oss-120b usedCyber range exercises usedExternal expert review of safety evaluation methodology used—
π0.5 usedInterleaved policy evaluation usedZero-shot evaluation in unseen environments used—
↑