Open Models Almanac
Methodsoptimizationlearning-rate schedule

single mention · optimization

Fixed elevated learning rate

The learning rate is held constant above its optimal value while the standard decay schedule is bypassed.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See learning-rate schedule for the methods filed beside it.

Filed under optimization › learning-rate schedule