Neural Scaling Law: A Brief Introduction

Table of ContentsIntroductionWhat is a neural network model?What is a neural scaling law?What are the types of neural scaling laws?Specific examples of neural scaling lawsReal-world applications of neural scaling lawsKey TakeawaysNeural Scaling Law predicts the relation between a property of a Deep Learning model to an attribute to number of parameters, size of training dataset and others.There are different variants of the law.One example is Jeff Dean's scaling law which states ac...