Working Backwards: Insights, Stories, and Secrets from Inside Amazon

by Colin Bryar

Morale is, in a sense, an output metric, whereas freedom to invent and build is an input metric. If you clear the impediment to building, morale takes care of itself.

In the 2016 shareholder letter, even though he wasn’t explicitly talking about two-pizza teams, Jeff suggested that “most decisions should probably be made with somewhere around 70% of the information you wish you had. If you wait for 90%, in most cases, you’re probably being slow. Plus, either way, you need to be good at quickly recognizing and correcting bad decisions. If you’re good at course correcting, being wrong may be less costly than you think, whereas being slow is going to be expensive for sure.”

Good examples like the Picking team demonstrated how longterm thinking, in the form of their up-front investments, generated compound returns over time. Later teams followed their lead. Sometimes it’s best to start slow in order to move fast.

After experimenting over many months across many teams, we realized that as long as we did the up-front work to agree on the specific metrics for a team, and we agreed on specific goals for each input metric, that was sufficient to ensure the team would move in the right direction. Combining them into a single, unifying indicator was a very clever idea that simply didn’t work.

Amazon’s SVP of Devices, Dave Limp, summed up nicely what might happen next: “The best way to fail at inventing something is by making it somebody’s part-time job.”6

another phrase you’ll hear at Amazon: be stubborn on the vision but flexible on the details.

An Amazon quarterly business review, for instance, might be broken down like this instead: Introduction Tenets Accomplishments Misses Proposals for Next Period Headcount P&L FAQ Appendices (includes things like supporting data in the form of spreadsheets, tables and charts, mock-ups)

Jeff has an uncanny ability to read a narrative and consistently arrive at insights that no one else did, even though we were all reading the same narrative. After one meeting, I asked him how he was able to do that. He responded with a simple and useful tip that I have not forgotten: he assumes each sentence he reads is wrong until he can prove otherwise. He’s challenging the content of the sentence, not the motive of the writer. Jeff, by the way, was usually among the last to finish reading.

Creating such solid narratives requires hard work and some risk-taking. Good ones take many days to write.

Finally, inevitably, he would ask, “Where are the mock-ups?” Jeff was referring to the visual representations that would show exactly how the new service would look on the Amazon website. Mock-ups should be detailed, showing the entire customer experience from landing page to purchase—screen design, buttons, text, the sequence of clicks, everything. To create a meaningful and informative mock-up you have to think through every element of what the service will offer, what the experience will be for the customer, how all the features will work on the page. It requires a ton of work to think through the whole business and a ton more work to create and refine the visuals.

A few weeks later we were back with rough mock-ups in hand. Jeff listened carefully to our presentation and then began asking detailed questions about every button, word, link, and color. For music, he asked how our service would be better than iTunes. For e-books, he wanted to know how much the e-books would cost. He asked if people would be able to read their e-books on a tablet or a phone as well as their PC. We answered as we had before. We hadn’t figured out all that stuff! We just needed his basic approval so we could hire the team, start negotiating deals with media companies, and get something launched. That answer did not go over well. At all. Jeff wanted to know exactly what we were going to build and how it would be better for customers than the competition. He wanted us to agree on those details before we started hiring a team or establishing vendor relationships or building anything.

We had freed ourselves of the quantitative demands of Excel, the visual seduction of PowerPoint, and the distracting effect of personal performance. The idea had to be in the writing.

We were working forward, trying to invent a product that would be good for Amazon, the company, not the customer. When we wrote a Kindle press release and started working backwards, everything changed. We focused instead on what would be great for customers. An excellent screen for a great reading experience. An ordering process that would make buying and downloading books easy. A huge selection of titles. Low prices.

The primary point of the process is to shift from an internal/company perspective to a customer perspective. Customers are pitched new products constantly. Why will this new product be compelling enough for customers to take action and buy it? A common question asked by executives when reviewing the product features in the PR is “so what?” If the press release doesn’t describe a product that is meaningfully better (faster, easier, cheaper) than what is already out there, or results in some stepwise change in customer experience, then it isn’t worth building.

The PR gives the reader the highlights of the customer experience. The FAQ provides all the salient details of the customer experience as well as a clear-eyed and thorough assessment of how expensive and challenging it will be for the company to build the product or create the service. That’s why it’s not unusual for an Amazon team to write ten drafts of the PR/FAQ or more, and to meet with their senior leaders five times or more to iterate, debate, and refine the idea.

Over time, we refined and normalized the specifications for the PR/FAQ. The press release (PR) portion is a few paragraphs, always less than one page. The frequently asked questions (FAQ) should be five pages or less. There are no awards for extra pages or more words. The goal isn’t to explain all the excellent work you have done but rather to share the distilled thinking that has come from that work.

However, restricting the length of the document is, to use a term that came up when describing the narratives, a forcing function—we have seen that it develops better thinkers and communicators.

These are the key elements of the press release: Heading: Name the product in a way the reader (i.e., your target customers) will understand. One sentence under the title. “Blue Corp. announces the launch of Melinda, the smart mailbox.” Subheading: Describe the customer for the product and what benefits they will gain from using it. One sentence only underneath the heading. “Melinda is the physical mailbox designed to securely receive and keep safe all your e-commerce and grocery deliveries.” Summary Paragraph: Begin with the city, media outlet, and your proposed launch date. Give a summary of the product and the benefit. “PR Newswire, Atlanta, GA, November 5, 2019. Today Blue Corp. announced the launch of Melinda, a smart mailbox that ensures secure and properly chilled delivery and storage for your online purchases and groceries.” Problem Paragraph: This is where you describe the problem that your product is designed to solve. Make sure that you write this paragraph from the customer’s point of view. “Today, 23 percent of online shoppers report having packages stolen from their front porch, and 19 percent complain of grocery deliveries being spoiled.” Solution Paragraph(s): Describe your product in some detail and how it simply and easily solves the customer’s problem. For more complex products, you may need more than one paragraph. “With Melinda, you no longer need to worry about getting your online purchases and deliveries stolen . . .” Quotes and Getting Started: Add ...

For internal FAQs, there is a more standardized list of topics you will need to cover. Here are some of the typical areas to address. Consumer Needs and Total Addressable Market (TAM) How many consumers have this need or problem? How big is the need? For how many consumers is this problem big enough that they are willing to spend money to do something about it? If so, how much money would they be willing to spend? How many of these consumers have the characteristics/capabilities/constraints necessary to make use of the product?

Economics and P&L What are the per-unit economics of the device? That is, what is the expected gross profit and contribution profit per unit? What is the rationale for the price point you have chosen for the product? How much will we have to invest up front to build this product in terms of people, technology, inventory, warehouse space, and so on?

Dependencies How will we convince couriers (USPS, UPS, FedEx, Amazon Fulfillment, Instacart, etc.) to actually use this device instead of their current/standard delivery methods? How will we ensure that couriers (who don’t work for you and over whom you have no control) will use the Melinda UI properly and bother to actually put packages in it instead of just leaving the package by the front door like they typically do? Won’t it take more time (which is precious) for them to make a delivery than it does today? What third-party technologies are we dependent on for Melinda to function as promised?

A common mistake among less-seasoned product managers is to not fully consider how third parties who have their own agendas and incentives will interact with their product idea, or what potential regulatory or legal issues might arise.

Feasibility What are the challenging product engineering problems we will need to solve? What are the challenging customer UI problems we will need to solve? What are the third-party dependencies we will need to solve? How will we manage the risk of the up-front investment required? These questions are intended to help the author clarify to the reader what level of invention is required and what kind of challenges are involved in building this new product.

The fact that most PR/FAQs don’t get approved is a feature, not a bug. Spending time up front to think through all the details of a product, and to determine—without committing precious software development resources—which products not to build, preserves your company’s resources to build products that will yield the highest impact for customers and your business.

Leadership and management are often about deciding what not to do rather than what to do.

Above all, keep in mind that the PR/FAQ is a living document. Once it is approved by the leadership team, it will almost certainly still be edited and changed (a process that should be directed by or reviewed with the leadership team). There is no guarantee that an idea expressed in an excellent PR/FAQ will move forward and become a product. As we’ve said, only a small percentage will get the green light. But this is not a drawback. It is, in fact, a huge benefit of the process—a considered, thorough, data-driven method for deciding when and how to invest development resources. Generating and evaluating great ideas is the real benefit of the Working Backwards process.

Jeff and I (Colin) once visited a Fortune 500 company to meet privately with the CEO in his office. During our meeting, an assistant dashed in and handed the boss a sheet of paper. The CEO glanced at it, waved it at us, and proudly said, “Our stock is up 30 cents this morning!” His mood brightened, as if he had personally caused the rise. As we drove to our next meeting, Jeff said, “There’s nothing that CEO did to cause that 30-cent blip in the stock price.” I agreed, and added that I wouldn’t be surprised if the assistant had thrown multiple printouts in the recycle bin that morning when the blip wasn’t so big. Would the same scene have played out if the share price had dropped 30 cents? The deeper lesson, one that we’ll explore in this chapter, is this: share price is what Amazon calls an “output metric.” The CEO, and companies in general, have very little ability to directly control output metrics. What’s really important is to focus on the “controllable input metrics,” the activities you directly control, which ultimately affect output metrics such as share price.

controllable input metrics,

Before you can improve any system . . . you must understand how the inputs affect the outputs of the system. You must be able to change the inputs (and possibly the system) in order to achieve the desired results. This will require a sustained effort, constancy of purpose, and an environment where continual improvement is the operating philosophy.2 Amazon takes this philosophy to heart, focusing most of its effort on leading indicators (we call these “controllable input metrics”) rather than lagging indicators (“output metrics”). Input metrics track things like selection, price, or convenience—factors that Amazon can control through actions such as adding items to the catalog, lowering cost so prices can be lowered, or positioning inventory to facilitate faster delivery to customers. Output metrics—things like orders, revenue, and profit—are important, but they generally can’t be directly manipulated in a sustainable manner over the long term. Input metrics measure things that, done right, bring about the desired results in your output metrics.

Identify the Correct, Controllable Input Metrics This step sounds easy but can be deceptively tricky, and the details matter. One mistake we made at Amazon as we started expanding from books into other categories was choosing input metrics focused around selection, that is, how many items Amazon offered for sale. Each item is described on a “detail page” that includes a description of the item, images, customer reviews, availability (e.g., ships in 24 hours), price, and the “buy” box or button. One of the metrics we initially chose for selection was the number of new detail pages created, on the assumption that more pages meant better selection. Once we identified this metric, it had an immediate effect on the actions of the retail teams. They became excessively focused on adding new detail pages—each team added tens, hundreds, even thousands of items to their categories that had not previously been available on Amazon. For some items, the teams had to establish relationships with new manufacturers and would often buy inventory that had to be housed in the fulfillment centers.

we asked ourselves, “If we work to change this selection metric, as currently defined, will it result in the desired output?” As we gathered more data and observed the business, this particular selection metric evolved over time from number of detail pages, which we refined to number of detail page views (you don’t get credit for a new detail page if customers don’t view it), which then became the percentage of detail page views where the products were in stock (you don’t get credit if you add items but can’t keep them in stock), which was ultimately finalized as the percentage of detail page views where the products were in stock and immediately ready for two-day shipping, which ended up being called Fast Track In Stock.

Fast Track In Stock, combined with inventory holding cost, provided the teams with an actionable and correct set of input metrics to add selection in a way that would profitably drive sales. Once you have metrics solidified, you can then set a standard and measure teams against that standard. For instance, we decided that in each category, we wanted 95 percent of detail page views to display a product that was in stock and ready for immediate shipping.

The right input metrics get the entire organization focused on the things that matter most. Finding exactly the right one is an iterative process that needs to happen with every input metric.

A big mistake people make is not getting started. Most WBRs have humble beginnings and undergo substantial changes and improvement over time.

But the Analyze stage is all about developing a comprehensive understanding of what drives your metrics. Until you know all the external factors that impact the process, it will be difficult to implement positive changes.

When Amazon teams come across a surprise or a perplexing problem with the data, they are relentless until they discover the root cause. Perhaps the most widely used technique at Amazon for these situations is the Correction of Errors (COE) process, based upon the “Five Whys” method developed at Toyota and used by many companies worldwide. When you see an anomaly, ask why it happened and iterate with another “Why?” until you get to the underlying factor that was the real culprit. This COE process requires the team who had a significant error or problem to write a document describing the problem or error, and to drill down on what caused it by asking and answering “Why?” five times in order to get to the true root cause. Charlie Bell, an SVP in AWS and a great operational guru at Amazon, put it aptly when he said, “When you encounter a problem, the probability you’re actually looking at the actual root cause of the problem in the initial 24 hours is pretty close to zero, because it turns out that behind every issue there’s a very interesting story.”

It is worth noting here that, at Amazon, even the most senior executives review the full WBR deck of metrics, including all the inputs and outputs. Metrics—as well as anecdotes about the customer experience—are the area where the leadership principle Dive Deep is most clearly demonstrated by senior leaders. They carefully examine the trends and changes in the metrics; audit incidents, failures, and customer anecdotes; and consider whether the input metrics should be updated in some way to improve the outputs.

We focus on variances and don’t waste time on the expected People like talking about their area, especially when they’re delivering as expected, and even more so when they exceed expectations, but WBR time is precious. If things are operating normally, say “Nothing to see here” and move along. The goal of the meeting is to discuss exceptions and what is being done about them. The status quo needs no elaboration.

Our business owners own metrics and are prepared to explain variances Amazon business owners are responsible for tracking the success of their area as defined by their metrics. In the weekly review, the owners, not the finance team, are expected to provide a crisp explanation for variances against expectations. As a result, business owners quickly become adept at spotting trends. Every week they review the deck before the WBR and respond by discussing what action they plan to take to address the variances. This is a hard-earned lesson; we’ve seen a metric owner display their metrics in front of a group where it’s obviously the first time that person has seen the data. That’s a big mistake, a waste of everyone else’s time, and will most definitely result in a kerfuffle with the senior leader in the room. By the time the WBR meeting occurs, each metric owner should have thoroughly analyzed the metrics they own.

We keep operational and strategic discussions separate The WBR is a tactical operational meeting to analyze performance trends of the prior week. At Amazon, it was not the time to discuss new strategies, project updates, or upcoming product releases.

While fear may be a good short-term motivator, it will ultimately cause more problems than it solves.

Mistakes should be a learning experience for all. If people become afraid of pointing out their own mistakes because they will feel humiliated in front of their peers, it’s human nature for them to do whatever they can to hide those mistakes in future meetings. Variances that get glossed over are lost learning opportunities for everybody. To prevent this, mistakes should be acknowledged as a chance to take ownership, understand the root cause, and learn from the experience. Some tension is unavoidable and appropriate, but we think it’s better to establish a culture where it’s not just okay, it’s actually encouraged to openly discuss mistakes.

Output Metrics Show Results. Input Metrics Provide Guidance.

Data and anecdotes make a powerful combination when they’re in sync, and they are a valuable check on one another when they are not.

Since then, we’ve gotten to be more mature, grounded in freedom from fear. Every time we do the awesome thing, of course we try to reward it. And the more a team eviscerates itself, being vocally self-critical, the more we try to reward that too. If a team is papering things over, and hasn’t looked at the customer experience, then you might ask hard questions.

Amazon’s approach to metrics embodies the Customer Obsession leadership principle. The relevance of Customer Obsession becomes evident in the company’s focus on input versus output metrics. If you look at the input metrics for Amazon, they often describe things customers care about, such as low prices, lots of available products, fast shipping, few customer service contacts, and a speedy website or app. A lot of the output metrics, such as revenue and free cash flow, are what you’d typically see in a company’s financial report. Customers don’t care about those.

In an interview after the Fire Phone was withdrawn, Jeff was asked about its failure and answered, “If you think that’s a big failure, we’re working on much bigger failures right now—and I am not kidding.”4 The magnitude of your inventions, and therefore your mistakes, needs to grow in lockstep with the growth of your organization. If it doesn’t, your inventions will likely not be big enough to move the needle.

As a company grows larger, it can become more difficult to keep the invention machine humming, and one impediment is “one-size-fits-all” decision-making. In the same 2015 shareholder letter, Jeff wrote, “Some decisions are consequential and irreversible or nearly irreversible—one-way doors—and these decisions must be made methodically, carefully, slowly, with great deliberation and consultation. If you walk through and don’t like what you see on the other side, you can’t get back to where you were before. We can call these Type 1 decisions. But most decisions aren’t like that—they are changeable, reversible—they’re two-way doors. If you’ve made a suboptimal Type 2 decision, you don’t have to live with the consequences for that long. You can reopen the door and go back through. Type 2 decisions can and should be made quickly by high judgment individuals or small groups.” Prime was a two-way door decision.

Over our long march to building Amazon’s digital business, we proved a powerful lesson: it takes exceptionally patient and unwavering leadership to persevere through the prolonged process of building a new business and navigating through transformative times in an established industry with entrenched interests. The fact that we entered as total beginners and emerged as industry leaders is in no small part a result of our adherence to being Amazonian in our principles and our way of thinking, including thinking big, thinking long-term, being obsessed with customers, being willing to be misunderstood for long periods of time, and being frugal—principles that few companies are capable of maintaining in the face of quarterly reporting requirements and the daily gyrations of the stock market.

In other words, his first action was not a “what” decision, it was a “who” and “how” decision. This is an incredibly important difference. Jeff did not jump straight to focusing on what product to build, which seems like the straightest line from A to B. Instead, the choices he made suggest he believed that the scale of the opportunity was large and that the scope of the work required to achieve success was equally large and complex. He focused first on how to organize the team and who was the right leader to achieve the right result.