Decision Level Fusion of Colour Histogram Based Classifiers for Clustering of Mouth Area Images Fahimeh Salimi ∗† Mohammad T Sadeghi ∗† ∗ Signal Processing Research Laboratory, Department of Electrical and Computer Engineering, Yazd University, Yazd, Iran † Engineering Research Centre, Yazd University, Yazd, Iran Email: salimi fahime@stu.yazduni.ac.ir, m.sadeghi@yazduni.ac.ir Abstract It is well known that in many situations combining diverse classifiers can improve the performance of a classification system. In this paper, a new histogram based lip seg- mentation technique is proposed considering local kernel histograms in different illumination invariant colour spaces. The histogram is computed in local areas using two Gaus- sian kernels; one in the colour space and the other in the spatial domain. Using the estimated histogram, the posterior probability associated to non-lip class is then computed for each pixel. This process is performed considering different colour spaces. A weighted averaging method is then used for fusing the posterior probability values. As the result a new score is obtained which is used for labelling the pixels as lip or non-lip. The advantage of the proposed method is that the segmentation process is totally unsupervised. So, the method is robust against different variations such as variation in lip shape, skin colour, facial hair, illumination, etc. Moreover, an improved performance is achieved by fusing colour information. 1. Introduction Real time lip tracking is an attractive research area in computer vision. The overwhelming interest in this topic stems from its numerous applications such as audio-visual speech recognition, audio-visual person identification, lip synchronisation and speech-based image coding. Lip tracking is a complex problem which involves many stages of processing. In the first stage the face of a subject has to be detected and its main facial features, including the mouth region, localised. After that the lip tracker is initialised by segmenting out the lip region pixels and detect- ing the boundary of this region. Then, it attempts to follow the changes in the boundary without necessarily performing segmentation. So, lip segmentation is an important stage in initialising the lip tracker systems [1]. However, accurate lip segmentation has proved to be difficult due to the weak colour contrast and the signif- icant overlap in colour features between the lip and the face regions and also due to the variation of lip shape in different peoples and skin colour in different human races. The presence of facial hairs and variation in illumination conditions also has negative impact on the performance of lip segmentation algorithms. Using unsupervised learning methods, where neither prior assumptions about the under- lying feature distribution nor training is needed, improves the capability of algorithm to deal with these problems [2]. In this paper we propose an unsupervised classification method for lip segmentation in colour images. This method is based on the local kernel colour histograms. This new type of histogram is introduced and tested for background subtraction in [3]. In [4], we applied a modified form of the method for segmentation of the mouth area images. The main difference between our method and the one proposed in [3], is that in our application we do not have an image as the reference image. So, a simple method was introduced for artificially generating the reference image. One of the main advantages of the proposed method is that since the method is unsupervised, it is robust against the variations in the shape and colour of the lip and skin regions. Moreover, variations due to make up, facial hair and illumination conditions do not highly affect the performance of the method. The method is also very fast. Therefore, it is suitable for real time applications. An important factor in colour image segmentation is the colour space used. It has been shown that in different ap- plications, different colour spaces could be more beneficial. Also, it is well known that in many situations combining the output of several classifiers can improve the classification accuracy. Therefore, in this study we wanted to see if we can improve the segmentation results by combining different colour spaces. The proposed colour kernel histogram based clustering algorithm is applied considering different colour spaces. A weighted averaging method [5] is then used for fusing the obtained results. Our experimental studies show that the performance of our histogram based method is better or comparable to some other state of the art algorithms. Moreover, the segmentation results slightly improve by fusing colour information. The rest of the paper is organised as follows. In Sec- International Conference on Digital Image Processing 978-0-7695-3565-4/09 $25.00 © 2009 IEEE DOI 10.1109/ICDIP.2009.80 416