Data Science for Business: What You Need to Know about Data Mining and Data-Analytic Thinking

by Foster Provost

for classification problems we can address all the issues by creating a formula that evaluates how well each attribute splits a set of examples into segments, with respect to a chosen target variable. Such a formula is based on a purity measure. The most common splitting criterion is called information gain, and it is based on a purity measure called entropy.