Multimodal Face Pair Matching System Ms. Divya K. Sawant, Prof. N. M. Shahane Department of Computer Engineering, K.K.Wagh Institute of Engineering, Education and Research,Nashik, Savitribai Phule Pune University, Maharashtra. divyasawant93@gmail.com, nmshahane@kkwagh.edu.in Abstract—A face recognition system can be thought as an identification or verification system. Face pair matching is a challenging task which aims to determine whether two face images represent the same person. Many times the face images being compared are having complicated facial variations and limited expressive information which becomes a difficult problem. To address these issues, the proposed approach concentrate on exploiting an additional set of face images called as cohort set. The proposed system is able to perform multimodal face pair matching using cohort information. The inputs to this system are pair of face images to be matched and the set of cohort face images. All cohort images are ranked separately based on pair of face images to generate two lists. Then cohort information are extracted from two sorted cohort list and combined with direct matching score of the two input face images to form modality. The final decision of matching is made by fusing all available face modalities. Index Terms—Cohort set, Face recognition, Multimodal fusion. I. I NTRODUCTION The recognition of human faces has been a long standing problem in computer vision and pattern recognition. A face recognition system can be thought as an identification system, a verification expert, or a pair matching system. There is a pre-enrolled face database in identification and verification system, but there is no pre-enrolled template database in face pair matching system. The role of identification system is to identify the given probe face image comes from which subject in the pre-enrolled database, while the role of verification system is to verify which person from among a set of people the picture represents, if any. Face pair matching system differs from these two systems. In this system, the input is pair of face images and the goal is to decide whether the face image represent same person or not. If they are from the same person, then it is called a genuine pair otherwise it is called an impostor pair.[1] Applications of face recognition are in law enforcement and video surveillance[2], HCI[3], demography estimation[4], smart cards, information security purpose. But among all these applications, automatically recognizing humans by analysing their faces has been one of the most difficult problems in computer vision and pattern recognition. This task is extremely hard because of limited expressive information i.e. only pho- tometric information is available in two face images. Facial descriptors have been devised in handling different sources of facial variations which includes patch-based LBP codes, learning-based descriptor, and discriminant face descriptor. Cosine similarity metric learning is a similarity measure for better matching. Single image per person require less effort for collecting, low cost for storing and processing, so one cannot have enough information to predict the result in test samples. This issue results in exploiting an additional set of samples called as cohort set. Utilizing information from cohort set increases performance of face pair matching task.[5] Complicated facial expressions are handled by face modal- ities like depth and thermal. For face pair matching task , fusing of different modalities results in reduction of diverse corrupting factors. Therefore the main focus is on multimodal cohort based face pair matching system. It includes : • Cohort list comparison for each modality is done through cohort identity list comparison (CILC) and cohort sample list comparison (CSLC). • Performance of system is improved by fusing different modalities. • Analysis of cohort list comparison is done. Fig. 1. Several face images of RGB, depth, thermal modalities resp. The paper is organized as follows. Section II describe existing cohort similar techniques related to face recognition. Section III describe system architecture. System analysis per- formance measures and datasets is describes in section IV. Partial results and comparison with similar system is describe in Section V. Section VI conclude the paper. II. REVIEW OF LITERATURE Many existing methods are available for face recognition using cohort information and different modalities. Some of them are listed below: Schroff et al. [6] proposed a data driven approach for face similarity. This approach is based on using Doppelganger list Sixth Post Graduate Conference for Computer Engineering (cPGCON 2017) Procedia International Journal on Emerging Trends in Technology (IJETT) IJETT | ISSN: 2455-0124( E) | 2350-0808 (P) | (IF: 0.456) | Volume 4 | Special Issue July - 2017 | 8292 Peer-review under responsibility of organizing committee of cPGCON 2017 © IJETT www.ijett.in