What accuracy doesn’t reveal are questions around failure. When a system fails, how are the errors distributed? We should not assume equal distribution of errors. Beyond just looking at the accuracies of a system, we can also learn more about performance by looking at the kinds of errors that are made. For error analysis, I took it one step further and looked at the results by each skin type, revealing even worse performance. When I looked at women with the darkest skin on the Fitzpatrick skin type scale, I found that for female-labeled faces of type VI, error rates were up to 46.8 percent. To
...more
