Visual Tracking and Servoing of Human Face for Robotic Head Amir-II A. A. Shafie, A. Iqbal, M. R. Khan Department of Mechatronics Engineering Kulliyyah of Engineering International Islamic University Malaysia Jalan Gombak, 53100 Kuala Lumpur, Malaysia aashafie@iiu.edu.my aseef.iqbal@gmail.com raisuddin@iiu.edu.my Abstract— In this paper, visual tracking and servoing of human face are implemented through image processing. A robotic humanoid head named Amir-II, equipped with web cam and servoing mechanism is used as the platform. The robotic head tracks the human face within the field-of-vision (FOV) while the servoing mechanism ensures the detected human face remains at the center of its FOV. The algorithm developed in this research utilizes the capability offered by scientific computing program MATLAB along with its Image Processing Toolbox. The algorithm basically compares the locations of the face in the image plane that is detected from the static face image captured from real-time video stream. The calculated difference is then used to produce appropriate motion command for the servo mechanism to keep track of the human face moving within the range of its FOV. Keywords - Humanoid head, Face Detection, Visual servoing, SMQT, split-up SNoW classifier, Matlab, Image Processing I. INTRODUCTION There are quite a number of researches going on around the globe involved in developing an effective Human-Robot Interaction (HRI) for socially interactive humanoid robots. To enable these robots engage in interaction with humans in social paradigm, the HRI plays a very important role [1, 2]. Rosalind [3] suggests that HRI for applications like socially interactive machines can be very effective if it can exchange emotional expressions with the human counterpart. The most common way of expressing emotional state of a human is via facial expression augmented with verbal cues and physical gestures. Some of the significant works in analyzing facial expressions are presented in [4, 5, and 6]. An important step towards developing an emotionally responsive and intelligent robot is the capability of using visual cue as an input and analyzing it when interacting with human operator. In this type of application, capability of detecting and tracking a human face from video sequences is necessary. But locating and tracking of human face from visual input is particularly challenging because human faces have a very high degree of variability in terms of pose, scale and important facial features. This paper discusses the techniques used in the humanoid head AMIR-II for detecting and tracking human face. II. EVOLUTION OF IIUM ROBOTIC HEAD A. First Prototype: Amir - I The first prototype of the robotic head, named AMIR-I [7, 8], had Basic Stamp 2 microcontroller at its heart. The controller was linked to 17 parallax servo motors connected with different parts of the mechanical structure. The aim of AMIR-I was to head-start into this emerging field of research and create a test bed for development and iterative improvement towards developing an interactive and facially expressive humanoid head. AMIR – I was capable of displaying only 3 basic emotions and valid head movements (pan-tilt) with its limited Degree-of-Freedom. AMIR-I had a PING))) ultrasonic sensor attached for identifying the presence of any operator in front and was only a platform to initiate the research on developing an effective human-robot interaction system. Figure 1: AMIR-II – The IIUM robotic head. International Conference on Computer and Communication Engineering (ICCCE 2010), 11-13 May 2010, Kuala Lumpur, Malaysia 978-1-4244-6235-3/10/$26.00 ©2010 IEEE