Escaping the Build Trap: How Effective Product Management Creates Real Value

by Melissa Perri

Strategic intents usually take a while to reach, on the magnitude of one to several years. Strategic intents are always aligned to the current state of the business. When determining what these intents are, the C-Suite of the company should ask, “What is the most important thing we can do to reach our vision, based on where we are now?” There should not be laundry list of desires or goals—​just a few key things that need to happen to make a big leap forward. Keeping the list of strategic intents small focuses everyone.

One intent is usually good for a small company, and three are plenty for a large organization. Yes, three. I know that sounds like very few goals for an organization of thousands of people, but that is key. This is also where the level and time frame matter. Strategic intents should be at a high level and business focused. They are about entering new markets, creating new revenue streams, or doubling down in certain areas.

The VP of product works with the product managers below them to determine which are the right problems to solve to achieve both of these things. Sometimes, one of the problems that should be solved does not relate directly to the product vision. This is where a company would decide to introduce a new product and to create a product portfolio.

The chief product officer (CPO) is responsible for setting the direction and overseeing the product portfolio. Having a philosophy for how your products or services help your company reach that vision in the near term or long term is key. To get there, the CPO answers these questions for their team: How do all of our products work as a system to provide value to our customers? What unique value does each of the product lines offer that makes this a compelling system? What overall values and guidelines should we consider when deciding on new product solutions? What should we stop doing or building because it does not serve this vision?

The team set out to pull the data, splitting up into two groups. One team was analyzing the acquisition funnel, looking at every step a user took from the time they got on the site until they signed up. There the team found very low conversion rate from people who went on the site to those who actually signed up and paid. “We see that we are doing well with marketing, but even with the discounts we already have, they aren’t signing up. How do we figure out what is stopping them? We don’t have any of their information,” said Monica. “Have you heard of a tool called Qualaroo?” asked Rich, the lead developer. “It allows us to poll people when they go toward the back button or try to leave the page. We can ask them what’s stopping them from signing up. I can easily add this to the site in about 10 minutes.”

After we have set the goal, we begin walking through the Product Kata. We ask ourselves the following: What is the goal? Where are we now in relation to that goal? What is the biggest problem or obstacle standing in the way of me reaching that goal? How do I try to solve that problem? What do I expect to happen (hypothesis)? What actually happened, and what did we learn?

It’s important to understand each phase so that we are doing just enough to get us to the goal. One of the biggest mistakes I’ve seen in product management is teams rushing in to apply a tool or practice at the wrong stage. Many times, they are experimenting unnecessarily when the problem is not yet known or when there is already a good idea about the solution.

When considering whether to experiment around a particular solution, I think of what my friend Brian Kalma, former head of UX for Zappos, once told me: “Don’t spend your time overdesigning and creating unique, innovative solutions for things that are not core to your value proposition. If someone has already solved that problem with a best practice, learn from that, implement their solutions, gather data to determine if it’s successful in your situation, and then iterate. Reserve your time and energy for the things that will make or break your value proposition.”

Retention is a lagging indicator, which is impossible to act on immediately. It will be months before you have solid data to show that people stayed with you. That is why we also need to measure leading indicators like activation, happiness, and engagement. Leading indicators tell us whether we’re on our way to achieving those lagging indicators like retention. To determine the leading indicators for retention, you can qualify what keeps people retained—​for example, happiness and usage of the product.

“What is the obstacle standing in our way that we want to tackle?” “We don’t understand enough about the problems the teachers are facing when creating courses.” “What is one step we can take to better understand this?” “User research,” said Matt. “I will line up 20 of our teachers for one-hour sessions and watch them create courses. In two weeks, I should have enough for us to identify the key pain points. Can you help me with the interviewing, Christa?” “Of course, let’s divide and conquer. Rich, can you sit in on a few so we can all be on the same page?” “Sure, I can come to about half of them this week. I’ll clear my schedule.”

Even if you have access to the people you need, customer research is not without its pitfalls. It can be tricky because, as you might have experienced already, people often immediately jump into telling you the solution. “Oh, I just need a button here that lets me do X,” they say. As a product manager, you need to back up and ask, “Okay, but why? Why do you need a button? Why do you think a button is the right thing? What are you trying to accomplish?” It is understanding the user’s need—not the button—that helps you to get closer to understanding the root of the problem.

Due to the misconception of this term, I have stopped using MVP altogether. Instead, I talk more about solution experimentation.

Because these are not designed to be long-lasting solutions, you want to limit exposure to your customers. With any experiment, it is important to think of how you will end it—to “close the loop.” Setting expectations on experiments with your customers is key to keeping them happy and to mitigating risk of a failed experiment. Explain to them why you are testing, when and how the experiment will end, and what you plan to do next. Communication is key to a successful experimentation process.

Concept testing is another solution experiment that focuses more on high-touch interaction with the customer. In this case, you try to demonstrate or show concepts to the user to gauge their feedback. These can vary in execution, from landing pages and low-fidelity wireframes to higher-fidelity prototypes or videos of how the service might play out. The idea here is to pitch the solution idea in the fastest, lightest way possible to convey the message.

If you want to make it evaluative, to firmly test a hypothesis, you need a definitive pass-or-fail criteria, when interviewing a customer about the concept. This can be what I call an ask—something you would need from the user, either in the form of a commitment, monetary value, time, or some other investment that shows they are interested.

The team put together a rough video, demonstrating what Dropbox could do. It had not built a demo or a prototype but instead used video editing to demonstrate what it would look and feel like to the investors. It felt like a real product demo, even though it wasn’t a finished product. When the investors saw it, they went wild. To them, it was magic. Dropbox was able to secure its funding and to validate that it was on the right path.

Internal tools are often neglected, but they still matter to the company. They need to be treated the same way as any other product. You need to understand the direction, diagnose the problem, learn more about it, and then learn what the right solution is. After you have experimented to prove value, you can concentrate on building your first version and optimizing.

There are many frameworks out there that will help you prioritize, like benefits mapping, Kano models, and others, but my favorite is Cost of Delay. If you understand the desired outcomes from a strategic perspective, you can use Cost of Delay to help you determine what to ship sooner. In his book, The Principles of Product Development Flow, Don Reinertsen talks about the importance of Cost of Delay in prioritizing work. He calls it “the one thing” that should be quantified. Cost of Delay is a numeric value that describes the impact of time on the outcomes you hope to achieve. It combines urgency and value so that you can measure impact and prioritize what you should be doing first.

We are done developing or iterating on a feature only when it has reached its goals. Often, teams ship a first version of the feature and then move on to the next, without measuring the outcomes for the user. As Jeff Gothelf, the author of Sense & Respond, once said, “Version 2 is the biggest lie in software development.” This mentality always leads to the build trap. Instead, teams should be working like the team at Marquetly, by setting the success criteria before launch, while measuring and iterating until they reach it. Version 1 should be looked at as a hypothesis, just like any other work. And, if we launch the feature and it is not hitting our goals, we need to be comfortable rolling it back and trying something else.

Even though our team was practicing the discovery process of product management with the right approach, we were in a silo, separated out into an innovation center without the proper resources to fully execute on what we were discovering. Kodak had to secure a new budget to put any of our plans in action, something it couldn’t do for another six months during its annual review. Its philosophy as an organization was not set up to succeed in the world of rapid innovation that emerged during the early 2000s.

If there is one main reason I have seen companies fail to make a transition, it’s the lack of leadership buy-in to move to an outcome-oriented company. Leaders will say that they want to achieve results, but, at the end of the day, they still measure success by features shipped. Why? There’s so much satisfaction in seeing things move, at both a leadership and a team level. People want to feel like they are accomplishing things. Checking off the boxes of finished work feels good, but we need to remember that this is not the only measure of success. So we need other ways to help us communicate and talk about progress at different levels.

Visibility in organizations is absolutely key. The more leaders can understand where teams are, the more they will step back and let the teams execute. Remember the strategic gaps from Chapter 11? The more you try to hide your progress, the wider that knowledge gap becomes. Leaders will demand more information and will crack down on your freedom to explore. If you keep things transparent, you will have more freedom to become autonomous.

During quarterly business review meetings, the senior leadership team, made up of the executives and the highest level of the organization, should be discussing progress toward the strategic intents and outcomes of a financial nature. This includes reviewing revenue for the quarter, churn of customers, and costs associated with development or operations. The chief product officer (CPO) and their VPs of product are responsible for communicating how the outcomes of product initiatives have furthered strategic intents, like Karen did at Marquetly. New strategic intents can be introduced in this meeting, as older ones are coming to completion. It is not a place to prioritize new product initiatives or to go into detail on them. That is what the product initiative review is for.

The product initiative review is another quarterly meeting that can be staggered with the quarterly business review on off months. This meeting is for the product development side of the house—CPO, CTO, design leaders, the VPs of product, and the product managers. Here we review the progress of the options against the product initiatives and adjust our strategy accordingly. This is the place for product managers to talk about the results of preliminary experimentation, research, or first releases, as they relate to overall goals. New product initiatives can be introduced in this meeting for feedback and buy-in, along with funding from the product development leadership group. Product teams can ask for more funding to build a first version or to optimize an existing solution.

Release reviews provide the opportunity for teams to show off the hard work they have done and to talk about success metrics. These should happen monthly, before features go out, to showcase what is in the pipeline to be released. During this meeting, we should be communicating only what we know is going to ship—​not experiments or research being conducted. Although not necessary, most executives like to attend this meeting to see what is being shipped out to customers. This can a...

As I mentioned in the first section of the book, it’s dangerous to be a sales-led organization because it can lead to a lack of alignment around strategy. But sales still needs something to sell. Creating working agreements and roadmaps that can be communicated to customers is key to developing a good relationship between product and sales. You can make an agreement with the sales team that anything being released as GA—​or anything further along in Beta—​can be added to its sales roadmap.

We call this team product operations. In growth stage companies, a chief of staff (under the CPO, like in Marquetly), runs it. In larger organizations, the product operations team still reports to the CPO but it needs an experienced leader, usually at the VP level, to oversee it. This team is in charge of streamlining all operational and process work that product teams need to be successful. This includes: Create automated and streamlined ways to collect data on progress toward goals and outcomes across teams. Report on goals, outcomes, roadmaps, progress, capacity, and costs across the product organization, translating these activities into financial implications for the company executives. Set up and maintain a product analytics platform to report on product engagement metrics across the organization. Standardize product processes that go across teams, such as strategy cadences, experimentation tracking and feedback, documentation on product features, collecting data, setting goals, creating and maintaining roadmaps, and sales enablement. Organize and run critical product meetings for strategy creation, strategy deployment, and releases. Conduct any coaching or training for the product teams.

The product operations team should be made up of a combination of project managers and product people. It’s good to allocate a few developers to this team, as well, so they can integrate with third parties, if needed, or build custom tools to fit a specific purpose.

When I was first interviewing the product leaders about how they were measured for success, they all laughed. “Do you want to know what we do in December? We stop everything we’re doing and look at the scorecard. If we haven’t delivered on the items there, because they were really not as important as we thought a year ago, we just start building anything that will satisfy those requirements. Melissa, we literally ship whatever shit we can build in time that will check that box. Come January, we spend all our time pulling apart all the sloppy code. But, hey, we all get our bonuses in March.”

This doesn’t mean that we should be failing in spectacular ways. With the rise of Lean Startup, we began to focus on outcomes, yes, but we also started to celebrate failure. I want to be clear here: it is not a success if you fail and do not learn. Learning should be at the core of every product-led organization. It should be what drives us as an organization. It is just better to fail in smaller ways, earlier, and to learn what will succeed, rather than spending all the time and money failing in a publicly large way. This is why we have problem and solution exploration in product management—​to de-risk failing in the market.

Many companies talk about how they want their people to be innovative and how they want to create crazy new products, but there has to be an understanding that it’s safe to fail in order to get innovation. When you don’t have safety built into your company, your product managers won’t feel comfortable trying something new. No one will.

This was the biggest shift for Marquetly. The company had to break out of its old way of budgeting once a year. Instead, it allocated funds to the product portfolio as a whole. Then it used the product initiative reviews to determine what should be funded, based on the amount of certainty toward the outcomes.

Who came up with the last feature or product idea you built? If I ask a product manager this question, I hope to see a look of confusion on his or her face. “What do you mean who came up with it? Well, our team did. Right? That’s how it normally works.” This kind of response is a sign of a healthy product management organization, in which management sets the goals and the team is given room to figure out how to reach them. The product manager should be leading the charge to discover user problems and to solve them.

What was the last product you decided to kill? Another sign of an unhealthy product management culture is the inability to kill a product or idea that will not help a company reach its goals. If you hear, “We never really kill anything,” it often means that there’s a pretty big problem.

When’s the last time you talked with your customers? What I dread hearing is, “Oh, well, management doesn’t really let us talk to customers. They’re worried about us annoying them too much.” Without a healthy dialogue between a company and its customers, there is no way to truly learn about what the customers want or need. An organization set up for success not only allows product managers to talk to customers, it encourages them to do so and recognizes this process as a huge part of the job.

What is your goal? This is the first question I ask any product manager during an interview process.1 If the product manager cannot articulate a clear goal, it’s a sign of poor product management at the organizational level. If the product manager does have a goal but it is more output centric than outcome focused, this also signifies an unhealthy product team.

What are you currently working on? A truly successful product manager talks more passionately about the problems the product development team is solving than the solutions they are shipping. This is one of the biggest signs of success, for me, and it goes hand-in-hand with the question of goals. When I ask product managers this question, I want to hear about what big problems they are tackling for the user and the business.

What are your product managers like? As product managers, we want to work in an organization where the role is respected and well regarded. I’ve seen many organizations where the product management function was not well-respected. There were two causes: product managers were either seen as too strong, or they were seen as too weak.