BIOLAB-ICAO: A NEW BENCHMARK TO EVALUATE APPLICATIONS ASSESSING FACE IMAGE COMPLIANCE TO ISO/IEC 19794-5 STANDARD Davide Maltoni, Annalisa Franco, Matteo Ferrara, Dario Maio, Antonio Nardelli C.d.L. Scienze dell’Informazione - Università di Bologna, via Sacchi 3, 47023 Cesena, ITALY DEIS – Viale Risorgimento, 2 – 40126 Bologna, Italy. E-mail: {davide.maltoni, annalisa.franco, matteo.ferrara, dario.maio, antonio.nardelli}@unibo.it ABSTRACT This work focuses on performance assessment of software applications designed to evaluate the compliance of a face image to the ISO/ICAO standards for machine readable travel documents. In this paper we describe the new large database (of compliant and non-compliant images) we gathered, the associated testing protocol and the preliminary results measured on some existing algorithms. Index Terms — ICAO, ISO/IEC 19794-5, face, machine readable travel documents. 1. INTRODUCTION Recent evaluation campaigns have clearly shown that face recognition algorithms provide satisfactory results only in the presence of images acquired under controlled pose and illumination. The achievement of reliable results in large- scale identification is thus subject to the availability of standardized, high-quality images. For the application of face recognition to Machine Readable Travel Documents, some rules and encoding formats were initially proposed by the International Civil Aviation Organization (ICAO) and successively adopted by the International Standard Organization (ISO). In fact, the ISO/IEC 19794-5 standard [1] specifies a record format for encoding, recording and transmitting the facial image information and defines scene constraints, photographic properties and digital image attributes of facial images. Although the standard provides some generic guidelines and several examples of acceptable/unacceptable face images, a clear, detailed and unambiguous description of all the requirements is still not available. In view of the widespread adoption of the new standard, some vendors of biometric technologies started to develop and distribute software applications (SDKs) able to automatically verify the compliance of a face image to the ISO standard. At today, no independent and systematic evaluation of these algorithms has been performed. A preliminary study conducted by some of the authors of this paper has been proposed in [2] where a set of 30 well defined characteristics, related to geometric (e.g., eye location and distance) and photographic (e.g., focus, contrast) properties of the face image has been defined(see Table 1). However, the database used in that work was still incomplete due to the lack of non-compliant images for many characteristics. Starting from the same set of characteristics and requirements already introduced in [2], a more in-depth evaluation is here carried out; in particular, two commercial software and a prototype developed in our laboratory have been tested on the new large database where all the characteristics are now represented. 2. DATABASE AND TESTING PROTOCOL 2.1. Database One of the main limitations of [2] was related to the database used for testing that did not include images non- compliant to the ISO standard for some characteristics (e.g., hat/cap, veil over face). That database has been significantly extended and all the possible image “defects”, as listed in Table 1, are now adequately covered. The new database (called BioLab-ICAO) consists of 7740 images gathered from different sources: 1) the whole AR Face dataset [3] (3314 images); 2) part of the images of the FRGC database [4] (2665 images); 3) some images internally acquired (943 images); 4) some “artificial” images (818 images). The artificial images have been generated, through image processing operations starting from some full compliant images, to cover “pixelation” and “washed out” defects. This was necessary because these defects were not present in the databases considered and these kind of images are difficult to acquire with a “working” standard camera. Throughout the database the image size varies from to . Each image has been manually labeled and some additional information is added: in particular, the compliance to each characteristic is manually indicated by using a three state logic (compliant, non- compliant and dummy 1 ) and some markers indicating the 1 The dummy state is used in case of uncertainty for a specific characteristics. 41 978-1-4244-5654-3/09/$26.00 ©2009 IEEE ICIP 2009