A/B Testing Quotes

“If you want to increase your success rate, double your failure rate. Thomas J. Watson, Former Chairman and CEO of IBM”
― Dan Siroker, A/B Testing: The Most Powerful Way to Turn Clicks Into Customers

“A mistake that some companies make is to start moving a bunch of levers around without clear planning upfront for what they're trying to optimize—and what will be impacted by those changes.”
― Dan Siroker, A/B Testing: The Most Powerful Way to Turn Clicks Into Customers

“Incrementalism can lead to local maxima. Be willing to explore to find the big wins before testing smaller changes and tweaks. Conversely, sometimes it's the incremental refinements that prove or disprove your hypotheses about what your users respond to. Use the insights from small tests to guide and inform your thinking about bigger changes. Consider entirely new alternative approaches to your principal business goals. Be willing to go beyond just testing “variations on a theme”—you might be surprised. If you're working on a major site redesign or overhaul, don't wait until the new design is live to A/B test it. A/B test the redesign itself.”
― Dan Siroker, A/B Testing: The Most Powerful Way to Turn Clicks Into Customers