Lean Analytics: Use Data to Build a Better Startup Faster (Lean (O'Reilly))

by Alistair Croll

Finding these engagement patterns means analyzing data in two ways: To find ways you might improve things, segment users who do what you want from those who don’t, and identify ways in which they’re different. Do the engaged users all live in the same city? Do all users who eventually become loyal contributors learn about you from one social network? Are the users who successfully invite friends all under 30 years old? If you find a concentration of desirable behavior in one segment, you can then target it. To decide whether a change worked, test the change on a subset of your users and compare that subset’s results to others. If you put in a new reporting feature, reveal it to half of your users, and see if more of them stick around for several months. If you can’t test features in this way without fallout — the customers who didn’t get the new feature might get angry — then at the very least, compare the cohort of users who joined after the feature was added to those who came before.