Open Models Almanac
Methodssoftware implementationkernel & quantization library

single mention ยท software implementation

ModelOpt AutoQuantize

ModelOpt AutoQuantize is a sensitivity-constrained mixed-precision search that replaces hand-authored layer rules for choosing quantization formats.1 For each quantizable module and candidate format it estimates a sensitivity from the quantization-induced output error weighted by the squared task-loss gradient, then assigns formats minimizing total sensitivity subject to a target effective bitwidth budget.2,3

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. to replace hand-authored layer rules with a sensitivity- constrained mixed-precision search โ€” NVIDIA Nemotron 3.5 Super and Lightning Technical Report
  2. min {๐‘ž๐‘–} โˆ‘๏ธ ๐‘– ๐‘†๐‘–(๐‘ž๐‘–)s.t. โˆ‘๏ธ ๐‘– ๐‘›๐‘–๐‘(๐‘ž๐‘–)โ‰ค ยฏ๐‘ โ€” NVIDIA Nemotron 3.5 Super and Lightning Technical Report
  3. ๐‘ is the target effective bitwidth โ€” NVIDIA Nemotron 3.5 Super and Lightning Technical Report

Also called ModelOpt AutoQuantize (AutoQuant).

Filed under software implementation โ€บ kernel & quantization library