Finally, let’s look at the trickiest situation, the fuzzy middle, where the hypothesis remains in an unproven state, neither true nor false, and your metrics are all over the place. Those are the hardest calls. This is the one actual kind of failure story for a team running an experiment. If your goal is to prove or disprove a hypothesis, and you can’t do either given a reasonable amount of time, then the experiment may actually be failing. This is why clarity of metrics is so important at the onset of an experiment.
