Skip to main content

One doc tagged with "int8"

View all tags

Quantization for Accelerators

An NPU's MAC array (see What Is an NPU?) is built around integer arithmetic first and floating point second, if at all. Getting a model onto that hardware efficiently means converting its weights and activations from floating point into integers in a way that a fixed set of scale and offset numbers can undo well enough that the model still works. That conversion — quantization — is a distinct engineering discipline from anything in ordinary model training, with its own vocabulary, its own failure modes, and its own tooling, and this page is that vocabulary.