Lean Analytics: Use Data to Build a Better Startup Faster (Lean (O'Reilly))

by Alistair Croll

Don’t sell what you can make; make what you can sell.

Lean isn’t about being cheap or small, it’s about eliminating waste and moving quickly, which is good for organizations of any size.

Now that it’s cheap, even free, to launch a startup, the really scarce resource is attention. A concierge approach in which you run things behind the scenes for the first few customers lets you check whether the need is real; it also helps you understand which things people really use and refine your process before writing a line of code or hiring a single employee.

Ratios are easier to act on. Think about driving a car. Distance travelled is informational. But speed — distance per hour — is something you can act on, because it tells you about your current state, and whether you need to go faster or slower to get to your destination on time.

A good metric changes the way you behave. This is by far the most important criterion for a metric: what will you do differently based on changes in the metric?

Vanity metrics might make you feel good, but they don’t change how you act. Actionable metrics change your behavior by helping you pick a course of action.

Leading metrics give you a predictive understanding of the future; lagging metrics explain the past. Leading metrics are better because you still have time to act on them

If you find a causal relationship between something you want (like revenue) and something you can control (like which ad you show), then you can change the future.

“Total active users” is a bit better — assuming that you’ve done a decent job of defining an active user — but it’s still a vanity metric. It will gradually increase over time, too, unless you do something horribly wrong.

The real metric of interest — the actionable one — is “percent of users who are active.” This is a critical metric because it tells us about the level of engagement your users have with your product. When you change something about the product, this metric should change, and if you change it in a good way, it should go up. That means you can experiment, learn, and iterate with it.

Number of followers/friends/likes. Counting followers and friends is nothing more than a popularity contest, unless you can get them to do something useful for you. Once you know how many followers will do your bidding when asked, you’ve got something.

Emails collected. A big mailing list of people excited about your new startup is nice, but until you know how many will open your emails (and act on what’s inside them), this isn’t useful. Send test emails to some of your registered subscribers and see if they’ll do what you tell them.