Skip to main content

One doc tagged with "distillation"

View all tags

Inference Optimization

Making the model cheap enough to serve, without quietly making it wrong. Every optimisation technique on this page trades accuracy, latency, or memory for one another — measure all three, before and after, or the "optimisation" is a guess.