Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing

by Ron Kohavi

While “statistical significance” measures how likely the result you observe or more extreme could have happened by chance assuming the null, not all statistically significant results are practically meaningful.