How to Get a Free Lunch: A Simple Cost Model for Machine Learning Applications

P. Domingos

This paper proposes a simple cost model for machine learning applications based on the notion of net present value. The model extends and unifies the models used in (Pazzani et al., 1994) and (Masand and Piatetsky-Shapiro, 1996). It attempts to answer the question Should a given machine learning system now in the prototype stage be fielded? The model’s inputs are the system’s confusion matrix, the cash flow matrix for the application, the cost per decision, the one-time cost of deploying the system, and the rate of return on investment. Like Provost and Fawcett’s (1997) ROC convex hull method, the present model can be used for decision-making even when its input variables are not known exactly. Despite its simplicity, it has a number of non-trivial consequences. For example, under it the no free lunch theorems of learning theory no longer apply.

This page is copyrighted by AAAI. All rights reserved. Your use of this site constitutes acceptance of all of AAAI's terms and conditions and privacy policy.