The Lean Product Playbook: How to Innovate with Minimum Viable Products and Rapid Customer Feedback

by Dan Olsen

Using more than 11 choices will overwhelm customers, while using fewer than 5 won't achieve enough granularity. For any bipolar scale, I recommend using an odd number of choices so that there is a neutral option in the middle.

Significant research has been performed on the reliability and validity of various scales, and it is generally agreed that 5-point scales are best for unipolar and 7-point scales are best for bipolar—which explains why I recommend the choices above.

The version to which I'm referring is based on calculating the “gap” between importance and satisfaction. So you simply take the rating for importance and subtract from it the rating for satisfaction.

Central to Ulwick's methodology is the idea that customers buy products and services to help them get a task or job done. Customers decide which product to buy based on how well it delivers their “desired outcomes” for the “job to be done.”

Must-have needs don't create satisfaction by being met. Instead, the need not being met causes customer dissatisfaction.

Delighters provide unexpected benefits that exceed customer expectations, resulting in very high customer satisfaction. The absence of a delighter doesn't cause any dissatisfaction because customers aren't expecting it.

A good product is designed with focus on the set of needs that are important and that make sense to address together. Swiss

One of the best definitions I've heard of strategy is: “deciding what you're not going to do.”

People think focus means saying yes to the thing you've got to focus on. But that's not what it means at all. It means saying no to the hundred other good ideas that there are.

Therefore, relevance became the most important benefit and the one that offered the biggest opportunity for differentiation. Google was able to achieve much higher relevance than other search engines due to its unique PageRank algorithm. Because they were best at the benefit that mattered most—and had comparable or better performance on the other dimensions—Google won the search engine wars.

What about delighters? Google Suggest, which automatically suggests search query matches, falls into this category.

In order to have a shot at beating the incumbent market leader, the value proposition for your new product would have to at least match them on the two important performance benefits of relevance and response time.

Even if Cuil had matched Google on those two performance benefits, they would have still needed a valued differentiator to gain significant market share.

To be strategic, you want to ensure that you are projecting forward in time, anticipating the important trends in your market and what competitors are likely to do.

For your MVP, you want to identify the minimum functionality required to validate that you are heading in the right direction.

I call this an MVP candidate instead of an MVP because it is based on your hypotheses. You haven't yet validated with customers that they agree that it is, in fact, a viable product.

You should be practicing divergent thinking, which means trying to generate as many ideas as possible without any judgment or evaluation. There will be plenty of time later for convergent thinking, where you evaluate the ideas and decide which ones you think are the most promising.

Well-written user stories usually follow the template: As a [type of user], I want to [do something], so that I can [desired benefit].

Independent: A good story should be independent of other stories. Stories shouldn't overlap in concept and should be implementable in any order.

Negotiable: A good story isn't an explicit contract for features.

Valuable: A good story needs to be valuable to the customer.

Estimable: A good story is one whose scope can be reasonably estimated.

Small: Good stories tend to be small in scope. Larger stories will have greater uncertainty, so...

Testable: A good story provides enough information to make it clear how to test that the story is “done”...

To start with, your MVP candidate needs to have all the must-haves you've identified.

After that, you should focus on the main performance benefit you're planning to use to beat the competition.

You should include your top delighter in your MVP candidate. That may not be necessary if you have a very large advantage on a performance benefit. The goal is to make sure that your MVP candidate includes something that customers find superior to others products and, ideally, unique.

At this point in the Lean Product Process, you have done a fair bit of work. You have: Formed hypotheses about your target customers Formed hypotheses about their underserved needs Articulated the value proposition you plan to pursue so that your product is better and different Identified the top feature ideas you believe will address those needs and broken them down into smaller chunks Prioritized those feature chunks based on ROI Selected a set of those feature chunks for your MVP candidate, which you hypothesize customers will find valuable

Once you have specified the feature set for your MVP candidate, you'll want to test it with customers. In order to do that, you need to create a user experience (UX) that you can show to customers, which is the top layer of the Product-Market Fit Pyramid.

Many people misinterpret the term MVP by placing too much emphasis on the word minimum. They use this as an excuse to build a partial MVP that has too little functionality to be considered viable by a customer.

Instead, the pyramid on the right shows that while an MVP has limited functionality, it should be “complete” by addressing those three higher-level attributes.

A landing page test that measures what percentage of prospective customers click the “sign up” button and leave their email addresses is focused on marketing, because there isn't any product functionality the customer can actually use.

Marketing tests can provide valuable learning, but they're not an actual product that creates customer value.

Qualitative means that you are talking with customers directly, usually in small numbers that don't yield statistical significance. Here, you care about the detailed information you learn from each individual test.

Quantitative research involves conducting the test at scale with a large number of customers. You don't care as much about any individual result and are instead interested in the aggregate results.

Quantitative tests are good for learning “what” and “how many”: what actions customers took and how many customers took an action

qualitative tests are good for learning “why”: the reasons behind different customers' decisions to take an action or not.

In general, when you are first starting to develop your product or marketing materials, it is most beneficial to start with qualitative tests to gain some initial understanding.

Marketing materials include anything you would want to put in front of a customer: a landing page, a video, an advertisement, an email, and so forth. This test is an attempt to understand how compelling they find this marketing material and why. You are not getting feedback on the product itself, but rather how you talk about and explain the product.

One of the most popular tests is the landing page or smoke test. In this test, you create a live web page to which you direct traffic.

The key metric that these tests measure is the conversion rate: the percentage of visitors to your landing page that clicked on the button to convert from a prospect to a customer.

There are two fundamentally distinct times when you can conduct qualitative product tests: before you've built your product and after you've built it. Both are valuable.

Typical product design deliverables include wireframes, mockups, and interactive prototypes—all of which are representations of what your product will be like without being the actual product itself.

Ideally, before building your MVP, you will have validated its design by testing increasingly higher fidelity artifacts with customers. When you iterate to a point where you feel that you have validated product-market fit with enough confidence, you would proceed with building a real MVP.

The fake door or 404 page test is a good way to validate demand for a new feature that you are considering building. The idea is to include a link or button for the new feature and see what percentage of customers click on it. This lets you gauge whether customers actually want the feature before you spend the resources to build it.

Of course, you want to be mindful of how long and how often you run fake door tests to avoid making your customers unhappy.

Usability focuses on the users' goals and the tasks they need to perform to achieve those goals. What percentage of users are able to successfully complete each task? What percentage are able to do so, but encounter problems along the way?

Beyond the successful completion of tasks, usability also includes efficiency. If customers are able to figure out how to use a feature, but it requires too many steps or takes too long, that results in poor usability.

You can mentally overwhelm users by showing them too much information or giving them too many choices. You can also tax their knowledge or memory. A great UX avoids requiring users to exert much physical or mental effort.