Open Models Almanac
Methods

single mention · not yet filed

Repeated test execution to detect flaky tests and reward instability

Repeated test execution is a supervision-quality check that reruns tasks' tests multiple times to verify reward outcomes stay stable.1 Test outcomes must hold across repeated runs, screening out flaky tests and environment-induced reward fluctuations before tasks are trusted.2

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. We check reward stability through repeated execution — MiMo V2 6 technical report
  2. We require these outcomes to remain stable across eight reruns, screening for flaky tests and environment- induced reward fluctuations — MiMo V2 6 technical report