Published in IET Computer Vision Received on 30th September 2010 Revised on 11th April 2011 doi: 10.1049/iet-cvi.2010.0166 Special Issue: Future Trends in Biometric Processing ISSN 1751-9632 Unobtrusive multi-modal biometric recognition using activity-related signatures A. Drosou 1,2 G. Stavropoulos 2 D. Ioannidis 2 K. Moustakas 2 D. Tzovaras 2 1 Department of Electrical Engineering, Imperial College London, London SW7 2AZ, UK 2 Informatics and Telematics Institute, Hellas 57001, Thermi-Thessaloniki, Greece E-mail: drosou@iti.gr Abstract: The present study proposes a novel multimodal biometrics framework for identity recognition and verification following the concept of the so called ‘on-the-move’ biometry, which sets as the final objective the non-stop authentication in an unobtrusive manner. Gait, that forms the major modality of the scheme, is complemented by new dynamic biometric signatures extracted from several activities performed by the user. Gait recognition is performed through a robust scheme that is based on geometric descriptors of gait energy images and is able to compensate for undesired gait behaviour like walking direction variations and stops. On the other hand, the biometric signatures, based on the user activities, are extracted by tracking of three points of interest and are seen to provide a powerful auxiliary biometric trait. Finally, score level fusion is performed and the experimental results illustrate that the proposed multimodal biometric scheme provides very promising results in realistic application scenarios. 1 Introduction Biometrics have recently gained significant attention from researchers, while they have been rapidly developed for various commercial applications ranging from surveillance and access control against potential impostors [1] to medical analysis purposes [2]. A number of approaches have been described in the past attempting to fulfil the different requirements of each application, such as reliability, unobtrusiveness, permanence etc. Generally speaking, biometric methods are categorised to physiological and behavioural [3], depending on the type of used features. On the one hand, physiological biometrics are based on both biological measurements and inherent characteristics of each individual. Fingerprint is a typical example of physiological biometric traits that is widely used in law enforcement for identifying criminals [4], whereas other recent applications are based on iris- [5] or face- identification [6]. Despite their high recognition performance, they all demonstrate a very restricted applicability to highly controlled environments. On the other hand, behavioural biometrics are related to specific actions and the way that each person executes them. They can potentially allow the non-stop (on-the- move) authentication or even identification in an unobtrusive and transparent manner to the subject and become part of an ambient intelligence (AmI) environment. Behavioural biometrics are the newest technology in the field biometrics and they have yet to be researched in detail. Even if physiological biometrics are considered more robust and reliable, behavioural biometrics have the inherent advantage of being less obtrusive [3, 7]. Recent work and efforts on human recognition have shown that human behaviour (e.g. extraction of facial dynamics features [8]) and motion (e.g. human body shape dynamics during gait [9]) provide the potential of continuous on-the- move authentication, when considering activity-related signals. 1.1 Related work Regarding gait recognition, significant advances have been lately achieved [9, 10]. Most of the recent gait analysis methods can be divided into two categories of complemental nature [11], the model-based and the feature- based ones. Model-based approaches study static and dynamic body parameters of the human locomotion [12], like stride length, stride speed and cadence [13]. A noise resistant method has been presented in [14], whereby the model-based gait signature is extracted by applying Fourier series and temporal trait gathering techniques. In general, model-based approaches [12–14] create models of the human body from the input gait sequences. Previous work on these approaches has shown that they can guarantee good degrees of view- and scale-invariance. However, experimental evaluation in larger, publicly available databases is still required, to compare their performance to that of feature- based methods. On the contrary, feature-based techniques do not rely on the assumption of any specific model of the human body for gait analysis. They usually employ simple methods, such as temporal correlation, linear time normalisation [15], full volumetric correlation on partitioned silhouette frames IET Comput. Vis., 2011, Vol. 5, Iss. 6, pp. 367–379 367 doi: 10.1049/iet-cvi.2010.0166 & The Institution of Engineering and Technology 2011 www.ietdl.org