More on this book
Community
Kindle Notes & Highlights
by
Marty Cagan
Read between
July 23 - August 12, 2018
writes up a short summary e‐mail of key learnings and sends it out to the product team. But forget big reports that take a long time to write, are seldom read, and are obsolete by the time they're delivered because the prototype has already progressed so far beyond what was used when the tests were done.
All of this is a long way of saying that good product teams spend most of their time on creating value. If the value is there, we can fix everything else. If it's not, how good our usability, reliability, or performance is doesn't matter.
You might experiment with pricing, positioning, and marketing, but you eventually conclude that this is just not a problem people are concerned enough about. The worst part of this scenario is that, in my experience, it's so easily avoided. The problem I just described can happen at the product level, such as an all‐new product from a startup, or at the feature level. The feature example is depressingly common. Every day, new features get deployed that don't get used.
The demand‐testing technique is called a fake door demand test. The idea is that we put the button or menu item into the user experience exactly where we believe it should be. But, when the user clicks that button, rather than taking the user to the new feature, it instead takes the user to a special page that explains that you are studying the possibility of adding this new feature, and you are seeking customers to talk to about this.
The best product companies—including Apple, Amazon, Google, Facebook, and Netflix—are great examples where this kind of innovation is institutionalized. In these companies, innovation is not something that just a few people get permission to pursue. It is the responsibility of all product teams.
That said, we need to do this in a responsible way. This really means doing two really big things—protect your revenue and brand, and protect your employees and customers.
I know this is a big claim, but I argue that qualitative testing of your product ideas with real users and customers is probably the single most important discovery activity for you and your product team.
Preparing a value test therefore includes preparing a usability test. I described how to prepare for and run a usability test in the last chapter, so for now let me again emphasize that it's important to conduct the usability test before the value test, and to do one immediately after the other.
The main challenge in testing value when you're sitting face to face with actual users and customers is that people are generally nice—and not willing to tell you what they really think. So, all of our tests for value are designed to make sure the person is not just being nice to you.
Keep in mind that this is a slightly different type of A/B test than optimization A/B testing.
In discovery A/B testing, we usually have the current product showing to 99 percent of our users, and the live‐data prototype showing to only 1 percent of our users or less. We monitor the A/B test more closely.
Do we know how to build this? Do we have the skills on the team to build this? Do we have enough time to build this? Do we need any architectural changes to build this? Do we have on hand all the components we need to build this? Do we understand the dependencies involved in building this? Will the performance be acceptable? Will it scale to the levels we need? Do we have the infrastructure necessary to test and run this? Can we afford the cost to provision this?
If you put an engineer on the spot, without time to investigate and consider, you are very likely to get a conservative answer, partly designed to make you go away.
The question isn't, “Can you do this?” Rather, you are asking them to look into it and answer the question, “What's the best way to do this and how long would it take?”
The last thing you want to have happen is that your team moves forward and takes the time to commercialize the solution and deliver a shippable product, only to find out that you can't ship because you are violating one of these constraints.
user test is when we test our product ideas on real users and customers.
A product demo is when you sell your product to prospective users and customers, or evangelize your product across your company.
The purpose is to show off the value of the prototype or product.
A walkthrough is when you show your prototype to a stakeholder and you want to make sure they see and note absolutely everything that might be a concern.
A discovery sprint is a one‐week time box of product discovery work, designed to tackle a substantial problem or risk your product team is facing.
Some people use the term design sprint rather than discovery sprint, but as the purpose of the work—when done well—goes significantly beyond design, I prefer the more general term.
There are several situations where I recommend a discovery sprint, starting with when the team has something big and critically important and/or difficult to tackle. Another situation where a discovery sprint helps is when the team is just learning how to do product discovery.
Discovery coaches are typically former product managers or product designers (or former leaders of these areas) and they have usually worked for, or with, leading product companies. So, they are able to work side by side with actual product managers and designers—not just reciting Agile platitudes, but showing the team how to work effectively.
Discovery coaches are not unlike Lean Startup coaches. The main difference is that Lean Startup coaches often focus on helping a team discover not only their product, but also their business model, and their sales and marketing strategy.
Rather than fight this reality, we can embrace it. One of the simplest techniques for facilitating moving to new ways of working is the use of pilot teams. Pilot teams allow the roll out of change to a limited part of the organization before implementing it more broadly.
Plan to continue with your existing roadmap process for six to 12 months. However, starting immediately, every time you reference a product roadmap item, or discuss it in a presentation or meeting, be sure to include a reminder of the actual business outcome that feature is intended to help.
If the feature you're working on is to add PayPal as a payment method, and the reason is to increase conversion, then be sure to always show the current conversion rate and the result you're hoping to achieve. Most important, after the feature goes live, be sure to highlight the impact on that conversion rate. If the impact was good, celebrate it. If the impact was not what was hoped, then emphasize to everyone that, while you did ship the feature, the result was not successful. Point out specifically what was learned, but explain that you have other ideas for ways to get the desired result.
It doesn't do anyone any good to build things that may work for the customer, but then at some review meeting find out that you're not allowed to deploy what was created.
it takes about two to three hours a week—meeting for half an hour or so with each key stakeholder—to keep them apprised, and to get their feedback on new ideas.
The second problem is that a group setting is not the forum for designing strong products. It results in design by committee, which yields mediocre results at best.
The first is to have a very strong and very intentional product culture, and to have that culture very well established so that new hires know they're joining a different type of company that takes pride in how they work and in using best practices. When you join Netflix, or Airbnb, or Facebook, it's something you figure out in your first days, and that's their intention.
The second way of preventing this is to make this explicit in the interview and onboarding process.
Good teams have a compelling product vision that they pursue with a missionary‐like passion. Bad teams are mercenaries. Good teams get their inspiration and product ideas from their vision and objectives, from observing customers' struggle, from analyzing the data customers generate from using their product, and from constantly seeking to apply new technology to solve real problems. Bad teams gather requirements from sales and customers. Good teams understand who each of their key stakeholders are, they understand the constraints that these stakeholders operate in, and they are committed to
...more
This highlight has been truncated due to consecutive passage length restrictions.
Customer‐centric culture.
Compelling product vision. By the time many companies reach scale, their original product vision is now largely realized, and the team is struggling to understand what's next.
Focused product strategy. One of the surest paths to product failure is to try to please everyone at once.
Strong product managers. The lack of a strong and capable product manager is typically a major reason for lack of product innovation.
Stable product teams.
Engineers in discovery.
Corporate courage. It's no secret that many companies become extremely risk averse as they grow larger.
Empowered product teams.
Product mindset. In an IT‐mindset organization, the product teams exist to serve the needs of the business. In contrast, in a product‐mind set organization, the product teams exist to serve the company's customers in ways that meet the needs of the business.
Time to innovate.
Technical debt.
Lack of delivery management.
Infrequent release cycles. Most teams with slow velocity have release vehicles that are too infrequent.
Not including engineers early enough during product discovery.
Changing priorities. Realize that rapidly shifting priorities cause significant churn and substantially reduces the total throughput and morale.
The Silicon Valley Product Group website (www.svpg.com) is designed as a free and open resource where we share our latest thoughts and learnings from the world of technology‐powered products. You will also find examples of the techniques described in the book (see www.svpg.com/examples) as well as a current recommended reading list (see www.svpg.com/recommended‐reading).
