JOURNAL OF INFORMATION SCIENCE AND ENGINEERING 21, 1121-1137 (2005) 1121 A Face and Speech Biometric Verification System Using A Simple Bayesian Structure ANDREW B. J. TEOH, S. A. SAMAD * AND A. HUSSAIN * Faculty of Information Science and Technology (FIST) Multimedia University 75450, Melaka, Malaysia E-mail: bjteoh@mmu.edu.my * Electrical, Electronic and System Engineering Department National University of Malaysia 43600, Bangi, Malaysia E-mail: {salina; aini}@eng.ukm.my Identity verification systems that use a mono modal biometric always have to con- tend with sensor noise and limitations of the feature extractor and matcher, while com- bining information from different biometrics modalities may well provide higher and more consistent performance levels. However, an intelligent scheme is required to fuse the decisions produced by the individual sensors. This paper presents a decision fusion technique for a bimodal biometric verification system that makes use of facial and speech biometrics. The decision fusion schemes considered have simple Bayesian structures (SBS) that particularize the univariat Gaussian density function, Beta density function or Parzen window density estimation. SBS has advantages in terms of computation speed, storage space and its open framework. The performances of SBS is evaluated and com- pared with that of other classical classification approaches, such as sum rule and Multi- layer Perceptron, on a bimodal database. Keywords: bimodal biometrics, face module, speech module, simple bayesian structure, decision fusion 1. INTRODUCTION In today’s electronically wired information society, there are more and more situa- tions which require an individual, as a user, to be verified by an electronic machine as in the case of transaction authentication on physical or virtual access control. Traditionally, these activities have mostly been conducted using ID numbers, such as a token or a password. The main problem with these numbers is that they can be used by unauthor- ized persons. On the other hand, biometric techniques use unique personal features of the user himself to verify the identity claimed. These techniques employ face, facial termo- gram, fingerprint, hand geometry, hand vein, iris, retinal pattern, signature, or voice print information. All these features have different degrees of uniqueness, permanence, meas- urability, user acceptability, performance, and robustness against circumvention [1]. However, there are some limitations to using just one biometric as the verification tool. For instance, it is estimated that 5% of the population does not have legible finger- Received April 28, 2003; revised August 11, 2003; accepted July 8, 2004. Communicated by Kuo-Chin Fan.