Open Models Almanac
Methodsevaluation

specific method · filed under evaluation

Controlled evaluation with fixed settings

Holds evaluation configurations, harnesses, resources, and other non-model settings constant to support fair comparisons.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See evaluation for the methods filed beside it.

sources
2
models
2
lab adopt it
1
strongest
used

Documented in

Evidence

2 quotes from 2 sources, one card per model, each opening with its strongest.

How sources treat it
used 2

All models were evaluated under the same evaluation settings including agentic resources (CPU compute and timeouts, tools, search APIs), input data, prompt templates, number of repeats, and extraction/metric implementation.

usedunclearin Nemotron 3.5 Super and evaluated baselinesNVIDIA
1 more from this model

all configs, harnesses, and other non-model parameters held static to ensure fair comparison that isolates the checkpoint impact

usedevaluation onlyin Nemotron 3.5 Super VLNVIDIA
↑