At Amazon, understanding what’s normal is the responsibility of the metrics owner, whether that’s an individual contributor or a manager of thousands. Many statistical methods, such as XMR control charts,3 can highlight when a process is out of control. For us, however, experience and a deep understanding of the customer most often turned out to be the best way to filter out the signal from the background noise. For the most part, metrics are reviewed daily by their owners and weekly in the WBR, so that expected fluctuations become familiar and exceptions stand out.
