Cross entropy method for optimization

In this article, we will understand the cross entropy method that is widely used as an optimization technique in machine learning.

Table of contentsWhat is entropy?The cross entropy methodApplications

What is entropy?

Before we get to know about cross entropy method, first we need to understand what is entropy. Entropy is defined as the level of 'surprise' in the variable's possible outcome.

Let us now talk about entropy and surprise in detail. Say we have a bag of 12 balls out of which 11...