Selection of optimized features and weights on face-iris fusion using distance images Maryam Eskandari a , Önsen Toygar b,⇑ a Department of Computer Engineering, Hasan Kalyoncu University, Gaziantep, Turkey b Department of Computer Engineering, Eastern Mediterranean University, Gazimag ˘usa, Northern Cyprus, Mersin 10, Turkey article info Article history: Received 20 July 2014 Accepted 23 February 2015 Available online xxxx Keywords: Multimodal biometrics Particle Swarm Optimization Backtracking Search Algorithm Information fusion Spoof attacks abstract The focus of this paper is on proposing new schemes based on score level and feature level fusion to fuse face and iris modalities by employing several global and local feature extraction methods in order to effectively code face and iris modalities. The proposed schemes are examined using different techniques at matching score level and feature level fusion on CASIA Iris Distance database, Print Attack face database, Replay Attack face database and IIIT-Delhi Contact Lens iris database. The proposed schemes involve the consideration of Particle Swarm Optimization (PSO) and Backtracking Search Algorithm (BSA) in order to select optimized features and weights to achieve robust recognition system by reducing the number of features in feature level fusion of the multimodal biometric system and optimizing the weights assigned to the face-iris multimodal biometric system scores in score level fusion step. Additionally, in order to improve face and iris recognition systems and subsequently the recognition of multimodal face-iris bio- metric system, the proposed methods attempt to correct and align the location of both eyes by measuring the iris rotation angle. Demonstration of the results based on both identification and verification rates clarifies that the proposed fusion schemes obtain a significant improvement over unimodal and other multimodal methods implemented in this study. Furthermore, the robustness of the proposed multimo- dal schemes is demonstrated against spoof attacks on several face and iris spoofing datasets. Ó 2015 Elsevier Inc. All rights reserved. 1. Introduction Currently, the identification and verification of human beings based on physical or behavioral characteristics is a trend in places with high security needs. In general, most biometric systems in the real time applications use a single biometric characteristic; uni- modal biometric is suffered due to different factors such as lack of uniqueness, non-universality and noisy data [1]. For instance, variations in terms of illumination, pose and expression lead to degradation of face recognition performance [1]. Performance of iris recognition can be degraded in non-cooperative situations [2]. In order to solve the problem raised by the single trait, multimodality that is extracting information from multiple bio- metric traits can be applied as a remedy and ultimately causes to improve the performance of biometric systems. In this study, face and iris biometrics are used to fuse the infor- mation because of many similar characteristics of these two modalities. Information fusion for multimodal biometric systems can be performed at four different levels: sensor level, feature level, matching score level and decision level fusion [1]. Due to the ease in accessing and combining the scores, matching score fusion level is more popular among all fusion levels and involves three differ- ent categories. The first category is Transformation-based score fusion where normalization of matching scores into a common domain is needed prior to combining due to incompatibility of dif- ferent modalities feature set. Classifier-based score fusion is the second category that concatenates the scores from different sys- tems. In fact, the scores from different classifiers are treated as a feature vector where each matching score is considered as an ele- ment of feature vector. Finally, the third category is Density-based score fusion that requires an explicit estimation of genuine and impostor matching score densities leading to an increase in imple- mentation complexity [3]. Current researchers’ studies [3,4] can be used as an evidence to state that employing score fusion tech- niques such as Sum Rule and Weighted-Sum Rule with a proper score normalization method leads to an unprecedented improve- ment on unimodal biometric systems performance. On the other hand, feature level fusion [43,44] considers the original feature sets of different modalities and therefore contains richer information http://dx.doi.org/10.1016/j.cviu.2015.02.011 1077-3142/Ó 2015 Elsevier Inc. All rights reserved. ⇑ Corresponding author at: Department of Computer Engineering, Eastern Mediterranean University, Gazimag ˘usa, T.R.N.C., Mersin 10, Turkey. Fax: +90 392 365 07 11. E-mail address: onsen.toygar@emu.edu.tr (Ö. Toygar). Computer Vision and Image Understanding xxx (2015) xxx–xxx Contents lists available at ScienceDirect Computer Vision and Image Understanding journal homepage: www.elsevier.com/locate/cviu Please cite this article in press as: M. Eskandari, Ö. Toygar, Selection of optimized features and weights on face-iris fusion using distance images, Comput. Vis. Image Understand. (2015), http://dx.doi.org/10.1016/j.cviu.2015.02.011