In this paper we argue that, despite decades of research in Artificial Intelligence, we do not evaluate Machine Learning models in the right way - neither in research papers nor in industrial deployments. We show why this is the case, propose concepts and metrics that allow us to properly assess the value of a model, and provide insights into what we look for in a "good" ML model.