Report on the Third Contest on Symbol Recognition Ernest Valveny 1 , Philippe Dosch 2 , Alicia Fornes 1 and Sergio Escalera 1 1 Computer Vision Center, Dep. Ci` encies de la Computaci´o Universitat Aut`onoma de Barcelona, Bellaterra (Spain) {ernest,afornes,sergio.escalera@cvc.uab.es} 2 LORIA, Universit´ e Nancy 2, Nancy, France {Philippe.Dosch@loria.fr} Abstract. In this paper we make a brief report of the third edition of the International Symbol Recognition Contest, organized in the context of GREC’07. This contest follows the series started at the GREC’03 workshop. In this report we describe the main changes introduced in the test data according to the conclusions of the past edition of the contest. We also summarize the results obtained by the only participant method. Finally, we point out some conclusions and open issues to be addressed in the next editions of the contest. 1 Introduction The performance evaluation of symbol recognition has been a focus of research interest in the last years. Several surveys on symbol recognition[1–4] pointed out the need of standard evaluation tools in order to compare the large number of symbol recognition methods. As a result, a generic framework for the evaluation of symbol recognition has been proposed [5]. In this framework, the main issues to be addressed by any performance evaluation system are identified (mainly, the generation of datasets and groundtruth, the definition of metrics, and the protocol of evaluation) and several alternatives are proposed and discussed in the special case of symbol recognition. Following this generic framework, and from a practical point of view, several contests have been organized. Actually, the first effort on the evaluation of sym- bol recognition was undertaken at ICPR’00 [6] where a contest was proposed using a dataset consisting of 25 electrical symbols, which were scaled and de- graded with a small amount of binary noise. Afterwards, the series of contests on symbol recognition in the context of the GREC workshop started in 2003. In the first edition [7], the dataset was composed of 50 architectural and elec- trical symbols. These symbols were rotated, scaled, degraded with binary noise and deformed through vectorial distortion in order to generate up to 72 differ- ent tests with increasing levels of difficulty and number of symbols. There were five methods participating in the contest. Then, in the second edition [8] some modifications were introduced according to the conclusions of the first contest.