We were doing this shopping search and saying, “We want a test where on this query we want to get 80 percent right answers.” And so they're saying, “Right! So if it's a wrong answer it's a bug, right?” And I said, “No, it's OK to have one wrong answer as long at it's not 80 percent.” So they say, “So a wrong answer's not a bug?” It was like those were the only two possibilities. There wasn't an idea that it's more of a trade-off.
Norvig
