This is an invitation to read the seminal research papers that have shaped the field of Deep Learning, alongside relevant chapters from books, blog posts, video explainers etc.
We can proceed in chronological order, starting all the way back from the introduction of backpropagation in the 80s that gave birth to multi-layer neural networks to the state-of-the-art, which as of January 2025, seems Deepseek-R1, or is it too soon to speak? ;)
This undertaking will thus be a walk through ideas that have been demonstrated to work at scale. Although one can never dismiss alternative paradigms entirely (e.g. evolutionary algorithms), lest a clever refinement (or just throwing tons of computational power) should resurrect them from the dead, rigorously understanding the Transformer based LLM model is crucial, given its total dominance currently.
We can proceed in chronological order, starting all the way back from the introduction of backpropagation in the 80s that gave birth to multi-layer neural networks to the state-of-the-art, which as of January 2025, seems Deepseek-R1, or is it too soon to speak? ;)
This undertaking will thus be a walk through ideas that have been demonstrated to work at scale. Although one can never dismiss alternative paradigms entirely (e.g. evolutionary algorithms), lest a clever refinement (or just throwing tons of computational power) should resurrect them from the dead, rigorously understanding the Transformer based LLM model is crucial, given its total dominance currently.