Donut

16%
Flag icon
There are two types of invariant metrics: 1. Trust-related guardrail metrics, such as expecting the Control and Treatment samples to be sized according to the configuration or that they have the same cache-hit rates. 2. Organizational guardrail metrics, such as latency, which are important to the organization and expected to be an invariant for many experiments. In the checkout experiment, it would be very surprising if latency changed.
Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing
Rate this book
Clear rating
Open Preview