Data Science for Business: What You Need to Know about Data Mining and Data-Analytic Thinking

by Foster Provost

Profit curves are appropriate when you know fairly certainly the conditions under which a classifier will be used. Specifically, there are two critical conditions underlying the profit calculation: The class priors; that is, the proportion of positive and negative instances in the target population, also known as the base rate (usually referring to the proportion of positives). Recall that Equation 7-2 is sensitive to p(p) and p(n). The costs and benefits. The expected profit is specifically sensitive to the relative levels of costs and benefits for the different cells of the cost-benefit matrix. If both class priors and cost-benefit estimates are known and are expected to be stable, profit curves may be a good choice for visualizing model performance. However, in many domains these conditions are uncertain or unstable.