The simple act of adding more layers, however, wasn’t a panacea—deeper networks demonstrated higher and higher accuracy scores at first, but soon reached a point of diminishing returns. As our ambitions pushed us to build bigger and bigger, we inadvertently turned neural networks into labyrinths, their excessive layering corrupting the signal along the journey from one end of the network to the other, halting the training process in its tracks and rendering the system useless.
