“The algorithm seemed to be really good at distinguishing the two rather similar canines; it turned out that it was simply labeling any picture with snow as containing a wolf. An example with more serious implications was described by Janelle Shane in her book You Look Like a Thing and I Love You: an algorithm that was shown pictures of healthy skin and of skin cancer. The algorithm figured out the pattern: if there was a ruler in the photograph, it was cancer.7 If we don’t know why the algorithm is doing what it’s doing, we’re trusting our lives to a ruler detector.”