The Singularity Is Nearer: When We Merge with AI

by Ray Kurzweil

When Minsky and Papert reached this conclusion, it effectively killed most of the funding for the connectionism field, and it would be decades before it came back. But in fact, back in 1964 Rosenblatt explained to me that the Perceptron’s inability to deal with invariance was due to a lack of layers. If you took the output of a Perceptron and fed it back to another layer just like it, the output would be more general and, with repeated iterations of this process, would increasingly be able to deal with invariance. If you had enough layers and enough training data, it could deal with an amazing level of complexity. I asked him whether he had actually tried this, and he said no but that it was high on his research agenda. It was an amazing insight, but Rosenblatt died only seven years later, in 1971, before he got the chance to test his insights. It would be another decade before multiple layers were commonly used, and even then, many-layered networks required more computing power and training data than was practical. The tremendous surge in AI progress in recent years has resulted from the use of multiple neural net layers more than a half-century after Rosenblatt contemplated the idea.