implementation detail · filed under evaluation
NeMo Evaluator SDK
The NeMo Evaluator SDK is used to collect evaluation results.
- sources
- 2
- model
- 1
- lab adopt it
- 1
- strongest
- used
How sources treat it
One count per evidence span, weakest treatment to strongest.
used 2
Documented in
Evidence
2 spans quoted from the sources, strongest treatment first.
All evaluation results were collected via Nemo Evaluator SDK
usedevaluation onlyin Nemotron 3 UltraNVIDIA
All evaluation results were collected via Nemo Evaluator SDK
usedevaluation onlyin Nemotron 3 UltraNVIDIA
Filed alongside
Other methods under evaluation :: evaluation harness.
DeepSeek Harness Minimal modeHarborAtomic binary rubric evaluationAutomatic in-flight checkpoint evaluation schedulingBenchmark evaluation frameworkCross-scaffold evaluationInline training evaluatorsIsolated container per rolloutJSON-structured answer-format promptmini-swe-agent harnessNeMo Gym and NeMo Evaluator-based harnessNeMo SkillsPool