Escaping the Build Trap: How Effective Product Management Creates Real Value

by Melissa Perri

In addition to vanity metrics, I often see product teams measuring output-oriented metrics, such as the number of features shipped, story points complete, or user stories worked on. Although these are good productivity metrics, they are not product metrics. They cannot tie the results of product development back to the business. So we need a set of metrics that can help us do that. There are many product frameworks available to help you think through the appropriate product goals. My two favorites are Pirate Metrics and HEART metrics.

Although data analysis is important, it can’t tell the entire story. So it’s essential that we all go talk to actual humans to get to the heart of their problems. In fact, Giff Constable wrote an entire book, called Talking to Humans,

User research, observations, surveys, and customer feedback are all tools that you can harness to better explore the problem from a user standpoint. User research, in this case, is not to be mistaken for usability testing, which involves showing a prototype or website and directing people to complete actions. There, you are learning whether they can use and navigate the solution easily, not whether the solution actually solves a problem. This type of research is called evaluative. Problem-based user research is generative research, meaning that its purpose is to find the problem you want to solve.

My friend Josh Wexler says, “Nobody wants to hear that their baby is ugly.” The way around this is to not get too attached. Kill the bad ideas before they take up too much time and energy from the teams and before you get hooked on them. Instead, fall in love with the problem you are solving.

With any experiment, it is important to think of how you will end it — to “close the loop.” Setting expectations on experiments with your customers is key to keeping them happy and to mitigating risk of a failed experiment. Explain to them why you are testing, when and how the experiment will end, and what you plan to do next. Communication is key to a successful experimentation process.

Concierge experiments deliver the end result to your client manually, but they do not look like the final solution at all.

The idea behind the Wizard of Oz is that, unlike the concierge experiment, it looks and feels like a real, finished product. Customers don’t know that, on the backend, it’s all manual. Someone is pulling the strings — just like the Wizard of Oz.

Concept testing is another solution experiment that focuses more on high-touch interaction with the customer. In this case, you try to demonstrate or show concepts to the user to gauge their feedback. These can vary in execution, from landing pages and low-fidelity wireframes to higher-fidelity prototypes or videos of how the service might play out. The idea here is to pitch the solution idea in the fastest, lightest way possible to convey

At a workshop recently, a product manager asked me, “Do we always need to run these experiments? What if it’s an easy problem to fix?” The answer is no. Although concierge, Wizard of Oz, and concept testing are all good techniques, sometimes you don’t need to experiment so heavily around multiple concepts. It is important to remember that these tools are used for higher amounts of uncertainty and, thus, larger risk in your solution ideas.

A North Star document explains the product in a way that can be visualized by the entire team and company. This includes the problem it is solving, the proposed solution, the solution factors that matter for success, and the outcomes the product will result in. North Stars are great for providing context to a wide audience. They should be evolved over time, as you learn more about your product. It’s important to note that this is not an action plan — it does not include how the team will be building the product. That is where story mapping comes in. Story mapping helps teams break down their work and align around goals. As Patton says, “Its purpose is to help the team communicate about their work and what needs to get done to deliver value.”

In his book, The Principles of Product Development Flow, Don Reinertsen talks about the importance of Cost of Delay in prioritizing work. He calls it “the one thing” that should be quantified. Cost of Delay is a numeric value that describes the impact of time on the outcomes you hope to achieve. It combines urgency and value so that you can measure impact and prioritize what you should be doing first.

The product-led organization is characterized by a culture that understands and organizes around outcomes over outputs, including a company cadence that revolves around evaluating its strategy in accordance to meeting outcomes. In product-led organizations, people are rewarded for learning and achieving goals. Management encourages product teams to get close to their customers, and product management is seen as a critical function that furthers the business.

In addition to reward structures that prevent people from innovating, the culture of the organization plays a big part. You might not be judging your teams for success based only on outputs, but they may still not be willing to try new things. Why? There may not be enough safety in the organization to fail and learn.

To really push boundaries, teams are going to have to try some perceivably wild stuff. It might not be the solution you originally thought of and the teams might not have all the answers at the beginning, but if they are not allowed to explore these weird paths, they will never push the status quo. The status quo is safe. The status quo keeps you from innovating.

Many companies talk about how they want their people to be innovative and how they want to create crazy new products, but there has to be an understanding that it’s safe to fail in order to get innovation. When you don’t have safety built in to your company, your product managers won’t feel comfortable trying something new. No one will.

Having the right communication, rewards, incentives, budgeting, policies, and safety are all important in an organization, but one more thing is still required to make you truly product-led. In addition to a culture that rewards and promotes learning, you need a culture that focuses on the customer. Many of the top companies today, such as Amazon, Netflix, Zappos, Dollar Shave Club, and Disney, have gotten where they are by focusing on the customer. You can see this attitude manifest in the way that executives talk about and treat their customers.