Status Updates From Neural Networks and Deep Le...

"In the early days of AI research people hoped that the effort to build an AI would also help us understand the principles behind intelligence and, maybe, the functioning of the human brain. But perhaps the outcome will be that we end up understanding neither the brain nor how artificial intelligence works!"
this cracked me up

Usually, when programming we believe that solving a complicated problem like recognizing the MNIST digits requires a sophisticated algorithm. But even the neural networks in the Wan et al paper just mentioned involve quite simple algorithms, variations on the algorithm we've seen in this chapter.

The lesson to take away from this is that debugging a neural network is not trivial, and, just as for ordinary programming, there is an art to it. You need to learn that art of debugging in order to get good results from neural networks. More generally, we need to develop heuristics for choosing good hyper-parameters and a good architecture.

i now know what
- a neural network is
- are preceptor neurons
- are sigmoid neurons
- a sigmoid function is and why sigmoid
- are feedforward neural networks and recurrent ones
- a cost function is (i think)
- how gradient descent helps us minimize our cost function and helps us get closer to finding the correct output
- a stochastic gradient descent is and what is on-line learning

no way this is free. anyway, pov: you are speedrunning learning neural networks

I love being a STEM major !!😍 (I’m currently being held at gun point)

Finished "Why are deep neural networks hard to train?". Mostly explaining the Vanishing Gradient problem.

Finished "A visual proof that neural nets can compute any function". Amazing chapter, easy to digest with all the visual helpers provided

"Improving the way neural networks learn" chapter finished. Excellent book.

Backpropagation chapter finished, really good!

Chapter 1 done

Finished Chapter 1: Using neural nets to recognize handwritten digits

Picked it up after almost 1 year, yay!

This book went on to explain various modifications that can be made to the vanilla neural network model to better fit certain problems. The elegance and flexibility of neural networks and deep learning continues to amaze and inspire me.

I got caught up with other hobbies and schoolwork and unfortunately diverted from this book for some time. But I recently dove back into it and learned more about the mechanism behind machine learning called backpropagation. The author skillfully makes a relatively complex subject seem intuitive and elegant. I look forward to expanding and applying my newfound knowledge.

The topic of neural networks and deep learning has always fascinated me, and I am delighted to have found a book that teaches it so well. So far I have learned the basic structure of a neural network and have been able to draw comparisons between neural networks and the human brain. I look forward to delving further into this topic.

end of chap 2, how backpropagation algorithm works

four fundamental equations of backpropagation

2 / 6: The four fundamental equations behind backpropagation - Plan of attack

2 / 6: The four fundamental equations behind backpropagation

2 / 6: The Hadamard product

2 / 6: The two assumptions we need about the cost function

2 / 6: Warm up: a fast matrix-based approach to computing the output from a neural network

1 / 6: Implementing our network to classify digits

1 / 6: Learning with gradient descent

Chapter 1 / 6 - The architecture of neural networks

Chapter 1 / 6: Sigmoid neurons