REPORT Recommendations for using the relative operating characteristic (ROC) Robert Gilmore Pontius Jr. • Benoit Parmentier Received: 23 July 2013 / Accepted: 31 December 2013 / Published online: 28 January 2014 Ó Springer Science+Business Media Dordrecht 2014 Abstract The relative operating characteristic (ROC) is a widely-used method to measure diagnostic signals including predictions of land changes, species distribu- tions, and ecological niches. The ROC measures the degree to which presence for a Boolean variable is associated with high ranks of an index. The ROC curve plots the rate of true positives versus the rate of false positives obtained from the comparison between the Boolean variable and multiple diagnoses derived from thresholds applied to the index. The area under the ROC curve (AUC) is a summary metric, which is commonly reported and frequently criticized. Our manuscript recommends four improvements in the use and inter- pretation of the ROC curve and its AUC by: (1) highlighting important threshold points on the ROC curve, (2) interpreting the shape of the ROC curve, (3) defining lower and upper bounds for the AUC, and (4) mapping the density of the presence within each bin of the ROC curve. These recommendations encourage scientists to interpret the rich information that the ROC curve can reveal, in a manner that goes far beyond the potentially misleading AUC. We illustrate the benefit of our recommendations by assessing the prediction of land change in a suburban landscape. Keywords Accuracy  AUC  Index  Land change  Map  Prediction  ROC  Threshold  Uncertainty Introduction Our manuscript offers recommendations for the use and interpretation of the relative operating character- istic (ROC), which is also known as the receiver operating characteristic (Swets 2010). The ROC is a quantitative method to compare a reference Boolean variable versus an index. We use the word ‘‘index’’ because index is general and has meaning in terms of ranking, while other authors have used alternative words such as: activation level, probability, propen- sity, likelihood, and suitability. An index displays higher values for observations that are deemed more likely for the presence of a Boolean feature. The reference Boolean variable shows presence versus absence of a feature, where each observation is usually coded as 1 for presence and 0 for absence. If the index value for an observation is greater than a threshold, then the observation is diagnosed as presence, otherwise the observation is diagnosed as absence for the particular threshold. Therefore, each threshold produces a binary diagnosis. Each diagnosis produces Electronic supplementary material The online version of this article (doi:10.1007/s10980-013-9984-8) contains supple- mentary material, which is available to authorized users. R. G. Pontius Jr.  B. Parmentier (&) Graduate School of Geography, Clark University, 950 Main Street, Worcester, MA 01610-1477, USA e-mail: benoit.parmentier@gmail.com R. G. Pontius Jr. e-mail: rpontius@clarku.edu 123 Landscape Ecol (2014) 29:367–382 DOI 10.1007/s10980-013-9984-8