Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing

by Ron Kohavi

First, we characterize the metric by understanding the baseline mean value and the standard error of the mean, in other words, how variable the estimate of our metric will be.