Ecological Modelling 261–262 (2013) 32–42 Contents lists available at SciVerse ScienceDirect Ecological Modelling jo ur nal home p ag e: www.elsevier.com/locate/ecolmodel A fuzzy set approach to assess the predictive accuracy of land use simulations Jasper van Vliet a,b,∗ , Alex Hagen-Zanker c , Jelle Hurkens a , Hedwig van Delden a a Research Institute for Knowledge Systems bv, Maastricht, The Netherlands b Institute for Environmental Studies and Amsterdam Global Change Institute, VU University, Amsterdam, The Netherlands c Civil and Environmental Engineering, University of Surrey, Guildford, Surrey, United Kingdom a r t i c l e i n f o Article history: Received 22 November 2012 Received in revised form 24 March 2013 Accepted 26 March 2013 Keywords: Land use model Accuracy assessment Fuzziness Map comparison Kappa statistics a b s t r a c t The predictive accuracy of land use models is frequently assessed by comparing two data sets: the sim- ulated land use map and the observed land use map at the end of the simulation period. A common statistic for this is Kappa, which expresses the agreement between two categorical maps, corrected for the agreement as can be expected by chance. This chance agreement is based on a stochastic model of random allocation given the distribution of class sizes. Two existing statistics extend Kappa to make it more appropriate for the assessment of land use models: Fuzzy Kappa uses fuzzy set theory to include degrees of similarity, which adds geographical nuance because it distinguishes between small and large disagreement in position and in land use classes. Kappa Simulation, on the other hand, addresses the stochastic model that underlies the expected agreement: when a model starts from an initial land use map and subsequently makes changes to it, a stochastic model of random allocation given the distribu- tion of class sizes has little relevance. The expected accuracy in Kappa Simulation is therefore based on transition probabilities relative to the initial map. This paper presents Fuzzy Kappa Simulation, a statistic that combines the geographical nuance of Fuzzy Kappa with the stochastic model of Kappa Simulation. This new statistic is demonstrated on a case study example and results are compared with other vari- ations of Kappa. The comparison confirms that Fuzzy Kappa Simulation is the only statistic to evaluate models in terms of land use transitions, while also being sensitive to geographical nuance. © 2013 Elsevier B.V. All rights reserved. 1. Introduction In the last decade, many land use models have evolved into tools that can be used to study land use change processes, conduct scenario studies or perform policy analyses for real world cases (Aljoufie et al., 2013; Hellmann and Verburg, 2011; Stanilov and Batty, 2011). Applying land use models for these purposes requires an understanding of their performance. This performance can at least partly be characterized by their predictive accuracy, which is often assessed from its capacity to reproduce historical land use changes. This is typically assessed by comparing the simulated land use map and the observed land use map at the pixel level. Sev- eral map-comparison methods exist for this, including the Kappa statistic (Monserud and Leemans, 1992), the Tau coefficient (Ma and Redmond, 1994), and the Average Mutual Information (Foody, 2006). ∗ Corresponding author at: Institute for Environmental Studies and Amsterdam Global Change Institute, VU University, Amsterdam, The Netherlands. Tel.: +31 20 59 83052. E-mail address: Jasper.van.vliet@vu.nl (J. van Vliet). Map comparison methods indicate for each pixel whether the land use is similar in both maps or not. Consequently, when a model simulates a particular land use change in the wrong location, it is registered as two errors: one change where it should not be and one non-change where it should be. However, from a modeller’s point of view, simulating the right change in nearly the right location may be considered as partially correct, while simulating this change at the other side of the study area would be a complete miss. Similarly, when a model simulates a land use change of a different nature than occurs in reality, it is traditionally registered as one error. However, some transitions may be considered a more severe error than oth- ers. For instance, a change from cropland to dense residential land simulated as a change from cropland into sparse residential land may be considered as partially correct, because both are transitions towards residential land. A crisp assessment of land use classes would therefore be unnecessarily harsh for the comparison of two maps (Foody, 2008), hence it can be meaningful to allow for spa- tial as well as thematic tolerance in the assessment of results of land use models. The use of fuzziness to interpret land use maps is further justified by uncertainties in land use data, including mixed pixels (Fisher et al., 2006; Foody, 2008) and uncertainty inherent to data acquisition techniques (Foody, 2002; Fritz and See, 2005). 0304-3800/$ – see front matter © 2013 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.ecolmodel.2013.03.019