Artificial neural networks are not adept at generalizing. An AI researcher may train one of their neural networks on tens of thousands of images so that it learns to classify them: “cars,” “gorillas,” “ice cream vans on fire (irony).” But deep neural networks have many layers of thousands of simple neuron-like units. So they have millions, tens of millions, of connections between those units, and the strength of every connection can be adjusted. Having far more connections to adjust than images to learn means artificial networks are prone to horrible overfitting;23 they learn the fine detail
...more
