Without explainability, we could easily obtain the results similar to those generated by Amazon’s internal AI system to screen job applications. That system was trained on ten years of the company’s hiring data, and because the tech industry is male dominated, the AI system taught itself to be sexist, ranking resumes lower if they included the word “women’s” or if the applicant graduated from an all-women’s college. (There are times when we don’t want the future to look like the past.)
