Working Backwards: Insights, Stories, and Secrets from Inside Amazon

by Colin Bryar

When you see an anomaly, ask why it happened and iterate with another “Why?” until you get to the underlying factor that was the real culprit.

Charlie Bell, an SVP in AWS and a great operational guru at Amazon, put it aptly when he said, “When you encounter a problem, the probability you’re actually looking at the actual root cause of the problem in the initial 24 hours is pretty close to zero, because it turns out that behind every issue there’s a very interesting story.”

Improve

Control

Once a process is well understood and the decision-making logic can be encoded in software or hardware, it’s a potential candidate for automation.

The WBR: Metrics at Work

We’ll talk first about how data presentation

Second, we’ll describe the meeting itself,

The Deck

let’s review some of the Amazon deck’s distinctive features:

The deck represents a data-driven, end-to-end view of the business.

It’s mostly charts, graphs, and data tables.

How many metrics should you review?

Emerging patterns are a key point of focus.

Graphs plot results against comparable prior periods.

Graphs show two or more timelines, for example, trailing 6-week and trailing 12-month.

Anecdotes and exception reporting are woven into the deck.

The Meeting

A well-run WBR meeting is defined by intense customer focus, deep dives into complex challenges, and insistence on high standards and operational excellence.

But if they don’t continue to focus on inputs, they lose control over and visibility into the tools that generate output results.

We use consistent and familiar formatting to speed interpretation

We focus on variances and don’t waste time on the expected

Our business owners own metrics and are prepared to explain variances

This is a hard-earned lesson; we’ve seen a metric owner display their metrics in front of a group where it’s obviously the first time that person has seen the data. That’s a big mistake, a waste of everyone else’s time, and will most definitely result in a kerfuffle with the senior leader in the room. By the time the WBR meeting occurs, each metric owner should have thoroughly analyzed the metrics they own.

Sometimes even the well prepared are hit with a question to which the right answer isn’t immediately apparent. In that case, the owner is expected to say something like, “I don’t know. We are still analyzing the data and will get back to you.” This is preferable to guessing, or worse, making something up on the fly.

We keep operational and strategic discus...

The WBR is a tactical operational meeting to analyze performance trends of the prior week. At Amazon, it was not the time to discuss new strategies, proj...

Variances that get glossed over are lost learning opportunities for everybody. To prevent this, mistakes should be acknowledged as a chance to take ownership, understand the root cause, and learn from the experience. Some tension is unavoidable and appropriate, but we think it’s better to establish a culture where it’s not just okay, it’s actually encouraged to openly discuss mistakes.

Anatomy of a Metrics Chart

As we noted above, at Amazon we routinely place our trailing 6 weeks and trailing 12 months side by side on the same x-axis. The effect is like adding a “zoom” function to a static graph that gives you a snapshot of a shorter time period, with the added bonus that you’re seeing both the monthly graph and the “zoomed-in” version of it simultaneously.

This graphic measures page views for a business, and conveys a lot of data in a small space:

Output Metrics Show Results. Input Metrics Provide Guidance.

Input vs Output Metrics

Not Every Chart Compares Against Goals

Data Combined with Anecdote to Tell the Whole Story

One example is a program called the Voice of the Customer.

CP is defined as the incremental money generated after selling an item and deducting the variable costs associated with that item.

Andon Cord.

You can speak about this

Jeff blurted out, “We need an Andon Cord for customer service.”

“the big red button”

Pitfall 1: Disaster Meetings

The Earn Trust leadership principle exists in part to prevent this behavior from occurring. It states, “Leaders listen attentively, speak candidly, and treat others respectfully. They are vocally self-critical, even when doing so is awkward or embarrassing. Leaders do not believe their or their team’s body odor smells of perfume. They benchmark themselves and their teams against the best.”

DMAIC

One Amazonian still recalls those disaster meetings even though they occurred more than 15 years ago. He said,

Many statistical methods, such as XMR control charts,3 can highlight when a process is out of control.

If you look at the input metrics for Amazon, they often describe things customers care about, such as low prices, lots of available products, fast shipping, few customer service contacts, and a speedy website or app.

Controllable input metrics are a quantitative (diving deep with data) and qualitative (anecdotes) way of measuring how well the organization is satisfying these customer interests so that the output metrics trend the way the company desires.

Introduction to Part Two

Now we come to the application—the proof that the elements that go into being Amazonian produce results.

In 2015 Jeff wrote, “We want to be a large company that’s also an invention machine. We want to combine the extraordinary customer-serving capabilities that are enabled by size with the speed of movement, nimbleness, and risk-acceptance...

The other key is frugality. You can’t afford to pursue inventions for very long if you spend your money on things that don’t lead to a better customer experience, like trade show booths, big teams, and splashy marketing campaigns.