Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing

by Ron Kohavi

In general, overpowering an experiment is fine and even recommended, as sometimes we need to examine segments (e.g., geographic region or platform) and to ensure that the experiment has sufficient power to detect changes on several key metrics.