Online Evaluation and A/B Testing
The offline metric improved, and the business metric did not. This gap is common enough to expect it as the default outcome, not a surprising exception — offline metrics are proxies, and only a genuine online experiment measures the thing actually being optimised for.