“A key point is that an increased number of attributes relative to training points provides additional degrees of freedom to the optimization problem, as a result of which irrelevant solutions become more likely. Therefore, a natural solution is to add a penalty for using additional features. Specifically we can add a penalty for each parameter w i, which is non-zero. One can express this penalty using the L 0-norm of the vector”