Open Models Almanac
Methodsevaluationevaluation harness

methods · evaluation

evaluation harness

21 methods filed at this node or below it, from the sources of 14 models.

In this branch 21

Everything filed at this node or below it, with one collapsible heading per child node.

filed here 21

NeMo Evaluator SDK used · 3 sources · 3 quotes
DeepSeek Harness Minimal mode used · 2 sources · 2 quotes
Harbor Framework used · 2 sources · 2 quotes
18 single mentions
Savanna Benchmark abstraction core · 1 source · 1 quote
Atomic binary rubric evaluation used · 1 source · 1 quote
Benchmark-specific evaluation recipes used · 1 source · 1 quote
Custom internal evaluation harness used · 1 source · 1 quote
Isolated-container-per-rollout execution used · 1 source · 1 quote
JSON answer-format prompting used · 1 source · 1 quote
LM Evaluation Harness used · 1 source · 1 quote
mini-swe-agent harness used · 1 source · 1 quote
NeMo Gym and NeMo Evaluator harness used · 1 source · 1 quote
NeMo Skills used · 1 source · 1 quote
Pool agent harness used · 1 source · 1 quote
Priority-ordered answer extraction parser used · 1 source · 1 quote
Tinker Cookbook benchmark framework used · 1 source · 1 quote
BenchmarkEvaluator optional · 1 source · 1 quote

By model

Which of this branch's methods each model's own documents describe, and how strongly. Under each model: its strongest treatment anywhere in the branch.

↑