Adagrad: An Adaptive Gradient Algorithm for Optimization

Table of Contents
I. Introduction
II. How does Adagrad work?
III. Advantages of Adagrad
IV. Applications of Adagrad
V. Code implementation
VI. Comparison with other algorithms
VII. Conclusion

Adaptive Gradient Algorithm, abbreviated as Adagrad, is a gradient-based optimization algorithm first introduced in 2011. The research paper that talks about it explains that Adagrad is designed to adapt the learning rate for each parameter during the optimization process, based on the past gradients observ...