Inference Optimization
Making the model cheap enough to serve, without quietly making it wrong. Every optimisation technique on this page trades accuracy, latency, or memory for one another — measure all three, before and after, or the "optimisation" is a guess.
Quantization for Accelerators
An NPU's MAC array (see What Is an NPU?) is built around integer arithmetic first and floating point second, if at all. Getting a model onto that hardware efficiently means converting its weights and activations from floating point into integers in a way that a fixed set of scale and offset numbers can undo well enough that the model still works. That conversion — quantization — is a distinct engineering discipline from anything in ordinary model training, with its own vocabulary, its own failure modes, and its own tooling, and this page is that vocabulary.