Lean Analytics: Use Data to Build a Better Startup Faster (Lean (O'Reilly))

by Alistair Croll

A good metric is comparative.

A good metric is understandable.

A good metric is a ratio or a rate.

A good metric changes the way you behave.

Qualitative versus quantitative metrics

The real metric of interest — the actionable one — is “percent of users who are active.”

Avinash Kaushik,

Steve Blank

That doesn’t mean the team shouldn’t have taken a guess. Without that initial line in the sand, they would have had no benchmark for learning, and Kyle might not have picked up the phone. But now he really understood his customers. The combination of quantitative and qualitative data was key.

Ash Maurya’s Lean Canvas,

River Out Of Eden

The most important question in the survey is “How would you feel if you could no longer use this product or service?” In Sean’s experience, if 40% of people (or more) say they’d be very disappointed to lose the service, you’ve found a fit, and now it’s time to scale.

Ash Maurya

“Bad data is a pain to sell to people. And even good data is a pain to sell to someone if it doesn’t actually help someone, whether that’s because it’s not in a form that helps them work out what to do or because it doesn’t actually answer

the questions they are asking,” he says. “But when the data’s good and it really does help someone, then nobody can refuse it.”

“If you really believe in the data and the recommendations that the data makes, then you focus on why the person doesn’t understand the data and you help them to understand it,” he explained. “When they understand, then their eyes light up, and they become a bigger fan of the data than I am!”

“We simply said: ‘Asking thousands of people what they think about something is better than not asking them, right?’ and we showed that we could do so at high quality and low cost, and we went for it. After the first set of data came back, people fell in love with it: it helped them and they loved that.”

But what surprised Boyle the most was how valuable the (relatively small) consumer research continued to be, even though the organization could use the Big Data hoard from billions of transactions. “Good data beats big data,” he concludes. “I am constantly surprised at how good it can be when done properly.”

EMI had a huge amount of data, and little idea of how to use it. Rather than mining existing data sets, the company conducted surveys, building a simpler, more specific set of information that executives could get comfortable with. Once the value of this smaller interview data was proven, it was easier to sell the value of a broader data-driven culture.

Just because you have a lot of data doesn’t mean you’re data-driven. Sometimes, starting from scratch with a small set of data collected to solve a specific issue can help make the case for using data elsewhere in the organization.

It’s also more likely to get executive sponsorship because the problem is bounded and constrained, whereas nobody knows what controversies are lurking in the larger amounts of “data exhaust” the organization has collected over the years.

Beforehand: Get Buy-in

Customers weren’t telling Netflix they wanted to stream videos, but their patterns of usage, computer adoption,

broadband deployment, and browsing told the company a need existed.

Rather, you can ask for a modest budget, build analytics into the product, and launch sooner for less money. You can then use the data and customer feedback you get, which is vanishingly cheap to collect given today’s technology, to plead your case based on actual evidence.

Consider, for example, the many meal pre-ordering tools on the market today. These mobile applications let you place an order from a food court restaurant, pay, and pick up at an agreed-upon time without waiting. The restaurants like them because they save precious time in the lunchtime rush, and the diners like them because they’re simple and buyers can browse the menu at their leisure. It’s like Uber for lunch. Now consider what would happen if McDonalds were to decide to compete by introducing an application. It might have franchise constraints, or regulations for restaurants located in airports, or state laws about disclosing caloric content. All of these would have to be part of the MVP.

Offsetting this, however, is the huge amount of market control the company has. It could promote the app by giving away three hamburgers for free to everyone who installed it. The company would make back the money quickly in saved time at the cash register, and have access to a new marketing channel and untapped analytical insight into its customers. Intrapreneurs need to factor these kinds of constraints and advantages into their MVP far more than independent startups do. What’s more, as people start using your MVP, you have to manage the beta process carefully. You may be interfering with existing deals in the sales pipeline, or creating more work for customer support. If so, you need to have approval for the rollout and the buy-in of stakeholders. If you’re launching an entirely new product line, you may even have to camouflage it so you don’t cannibalize existing markets until you know it’s successful. This, of course, undermines your ability to use unfair advantages like an existing customer base.

Ultimately, the intrapreneur must manage the relationship with the host organization as well as the relationship with the target market. Initially, this can be intentionally distant, but as the disruptive product becomes part of the host, the handoff must be graceful.