Ecological Modelling 221 (2010) 1995–2002 Contents lists available at ScienceDirect Ecological Modelling journal homepage: www.elsevier.com/locate/ecolmodel Review Ecological relevance of performance criteria for species distribution models Ans M. Mouton a,b,c,∗ , Bernard De Baets b , Peter L.M. Goethals a a Laboratory of Environmental Toxicology and Aquatic Ecology, Ghent University, J. Plateaustraat 22, B-9000 Ghent, Belgium b KERMIT: Research Unit “Knowledge-based Systems”, Ghent University, Coupure links 653, B-9000 Ghent, Belgium c Research Institute for Nature and Forest, Kliniekstraat 25, 1070 Brussels, Belgium article info Article history: Received 5 November 2009 Received in revised form 23 April 2010 Accepted 27 April 2010 Available online 27 May 2010 Keywords: Prevalence Omission/commission errors CCI Kappa TSS abstract Species distribution models have often been developed based on ecological data. To develop reliable data-driven models, however, a sound model training and evaluation procedures are needed. A cru- cial step in these procedures is the assessment of the model performance, with as key component the applied performance criterion. Therefore, we reviewed seven performance criteria commonly applied in presence–absence modelling (the correctly classified instances, Kappa, sensitivity, specificity, the normalised mutual information statistic, the true skill statistic and the odds ratio) and analysed their application in both the model training and evaluation process. Although estimates of predictive perfor- mance have been used widely to assess final model quality, a systematic overview was missing because most analyses of performance criteria have been empirical and only focused on specific aspects of the performance criteria. This paper provides such an overview showing that different performance criteria evaluate a model differently and that this difference may be explained by the dependency of these cri- teria on the prevalence of the validation set. We showed theoretically that these prevalence effects only occur if the data are inseparable by an n-dimensional hyperplane, n being the number of input variables. Given this inseparability, different performance criteria focus on different aspects of model performance during model training, such as sensitivity, specificity or predictive accuracy. These findings have impor- tant consequences for ecological modelling because ecological data are mostly inseparable due to data noise and the complexity of the studied system. Consequently, it should be very clear which aspect of the model performance is evaluated, and models should be evaluated consistently, that is, independent of, or taking into account, species prevalence. The practical implications of these findings are clear. They provide further insight into the evaluation of ecological presence/absence models and attempt to assist modellers in their choice of suitable performance criteria. © 2010 Elsevier B.V. All rights reserved. Contents 1. Introduction .......................................................................................................................................... 1995 2. Performance criteria in presence-absence modelling ............................................................................................... 1996 3. Application of performance criteria for model evaluation .......................................................................................... 1997 4. Application of performance criteria for model training ............................................................................................. 1998 4.1. Analysis of CCI ................................................................................................................................ 1998 4.2. Analysis of Kappa ............................................................................................................................. 1999 4.3. Analysis of TSS ................................................................................................................................ 1999 5. Towards adaptive model evaluation ................................................................................................................. 2000 6. Conclusions .......................................................................................................................................... 2001 Acknowledgement ................................................................................................................................... 2001 Appendix A. Supplementary data ................................................................................................................. 2001 References ........................................................................................................................................... 2001 ∗ Corresponding author at: Laboratory of Environmental Toxicology and Aquatic Ecology, Ghent University, J. Plateaustraat 22, B-9000 Ghent, Belgium. Tel.: +32 9 264 39 96; fax: +32 9 264 41 99. E-mail address: Ans.Mouton@UGent.be (A.M. Mouton). 1. Introduction In past decades, species distribution models have increasingly received attention due to their wide management applications in the context of biogeography, conservation biology and climate 0304-3800/$ – see front matter © 2010 Elsevier B.V. All rights reserved. doi:10.1016/j.ecolmodel.2010.04.017