Hacking Growth: How Today's Fastest-Growing Companies Drive Breakout Success

by Sean Ellis

ANALYSIS AND LEARNING The analysis of the test results should be conducted by either the analyst or the growth lead, if he or she has the expertise. The results should be written up in a test summary, and this document should include • The name and description of the test including the variants used and the target customers (for example, was the test run in a marketing channel, only toward mobile users, or to paid subscribers, etc.?) • The type of test run: was it a test of a product function, a change to marketing copy on a website page or screen in a mobile app, a creative test, or a new marketing tactic deployed? • Features it impacted: this could include the screenshots of where the test was run on the site, or on the screen of a mobile app, or include a copy of the creative from a particular billboard ad, TV, or radio ad experiment • Key metrics: what were the metrics that you were trying to improve with the test? • Test timing, including start and end dates of the test including day of week • The hypothesis of the test and the results, including the original ICE score, sample sizes, statistical confidence, and statistical power • Potential confounding issues, such as the time of year the test was run, or if there were other promotions that may have skewed visitor behavior • The conclusions drawn.