Use of metatruth image concept to assess forest change detection accuracy at pixel level K. NACKAERTS*, K. VAESEN, I. LIZARRAGA, B. MUYS and P. COPPIN Laboratory for Forest, Nature and Landscape Research, Vital Decosterstraat 102, 3000 Leuven, Belgium (Received 18 October 2001; in final form 21 August 2003 ) Abstract. Traditionally, the validation of a classified multispectral image only quantifies its correspondence to ground reference data containing thematic information generalized at the stand level, with stands represented as vector polygons. Little is known of the accuracy of such classifications at a scale below the stand. This study presents a methodology to assess classification accuracy at pixel level, i.e. sub-polygon, where the classification procedure is embedded in a change detection environment. A new type of reference data (Metatruth Image) was generated based on the integration of the outputs of various independent change detection procedures. The integration consisted of calculating for each pixel a probability distribution or pixel purity index for each change class by independent change detection procedures, defined by the number of times the pixel has been classified as a certain change class. First, the relationship between purity and accuracy was successfully validated. Next, the Metatruth Image was created based on ‘high purity pixels’. Performing traditional accuracy assessment on the outputs of individual change detection procedures using the Metatruth Image as reference dataset, demonstrated that former outputs identified change events accurately at pixel level. As a consequence, traditional accuracy assessment at polygon level underestimates the true accuracy at pixel level of the change detection procedure in a systematic way with differences in kappa coefficients of agreement around 20%. 1. Introduction Thematic maps are used in a wide variety of fields and applications, ranging from geology over climatology to forest management. Over the past several years, interest in the accuracy of maps has grown and with it research on the accuracy assessment of thematic maps. In a first approach, the global measures of error are based on the confusion matrix as the overall accuracy (Lunetta et al. 1991), the kappa coefficient of agreement (Rosenfield and Fitzpatrick-Lin 1986), the user’s accuracy and producer’s accuracy (Story and Congalton 1986). One advantage of these methods is that they yield a single overall thematic map accuracy index, usually presented as the fraction or percentage of correctly classified pixels. The kappa statistic was originally developed by Cohen (1960) to measure the observed International Journal of Remote Sensing ISSN 0143-1161 print/ISSN 1366-5901 online # 2004 Taylor & Francis Ltd http://www.tandf.co.uk/journals DOI: 10.1080/0143116032000160453 *Corresponding author; e-mail: kris.nackaerts@agr.kuleuven.ac.be INT. J. REMOTE SENSING, 20 JULY, 2004, VOL. 25, NO. 14, 2713–2723