Prediction Machines: The Simple Economics of Artificial Intelligence

by Ajay Agrawal

For every bomber that returned from bombing raids over Germany, the engineers could see where they had been hit by antiaircraft fire. The bullet holes in the planes were their data. But were these the obvious places to better protect the plane? They asked statistician Abraham Wald to assess the problem. After some thought and some rather thorough mathematics, he told them to protect the places without bullet holes. Was he confused? That seemed counterintuitive. Didn’t he mean to protect the areas of the plane that did have bullet holes? No. He had a model of the process that generated the data. He recognized that some bombers did not come back from the raids and conjectured that these bombers got hit in places that were fatal. In contrast, the bombers that made it home were hit in places that were not fatal. With this insight, the air force engineers increased the armor in the places without bullet holes, and the planes were better protected.6 Wald’s insight about the missing data required an understanding of where the data came from; given that the problem had not arisen before, the engineers did not have prior examples to draw from. For the foreseeable future, such calculations are beyond the abilities of prediction machines.