Improved Likelihood Function in Particle-based IR Eye Tracking Dan Witzner Hansen 1 Riad Hammoud 2 Ronald Satria 1 Jakob Sørensen 1 Dept. of Innovation 1 Delphi Electronics and Safety 2 IT University, Copenhagen M/C E110, P.O. Box 9005 DK-2300 Copenhagen Kokomo, Indiana 46904-9005 {witzner,leon98usdk,vanity}@itu.dk riad.hammoud@delphi.com Abstract In this paper we propose a log likelihood-ratio func- tion of foreground and background models used in a par- ticle filter to track the eye region in dark-bright pupil im- age sequences. This model fuses information from both dark and bright pupil images and their difference image into one model. Our enhanced tracker overcomes the issues of prior selection of static thresholds during the detection of feature observations in the bright-dark difference images. The auto-initialization process is performed using cascaded classifier trained using adaboost and adapted to IR eye im- ages. Experiments show good performance in challenging sequences with test subjects showing large head movements and under significant light conditions. 1. Introduction As one of the most salient features of the human face, eyes play an important role in interpreting and understand- ing a person’s desires, needs, and emotional states. In addi- tion, the unique geometric, photometric, and motion char- acteristics of the eyes also provide important visual cues for face detection, face recognition, and for facial expres- sion understanding. Robust non-intrusive eye detection and tracking is therefore crucial for human computer interac- tion, attentive user interfaces, and understanding human af- fective states and is gaining importance outside laboratory experiments and even found in domestic appliances. For example eye tracking is used for driver fatigue and behav- ior [12], eye typing and in connection with rendering digital displays [2]. Direct eye detection and tracking methods search for eyes without prior information about face location, and can further be classified into passive and active methods. Pas- sive eye detectors work on use images taken in natural scenes, without any special illumination. Good light con- ditions often lead to greater success and less effort on al- gorithm research and development. In the vision-based eye tracking methods, it is found that the use of near infrared light (reflected infrared) can be very rewarding in terms of ease and efficiency. This includes practically all stages in the eye tracker, starting from the detection, to tracking and gaze estimation. Active eye-detection and tracking methods employ special IR illumination. One of their limitations is that they are mainly applicable in controlled indoor envi- ronments as the amount of IR light in outdoor scenes may seriously disturb the image observations. When IR light falls on the eye, part of it is reflected back, through the pupil, in a tiny ray pointing directly towards the light source. When a light source is located close to the op- tical axis of the camera (on-axis light), the captured image shows a bright pupil. This effect is similar to the the red-eye effect when using flashlight in photography. When a light source is located away from the camera optical axis (off- axis), the image shows a dark pupil. However, neither of these light schemes solely allow for robust results, as there are also other bright and dark objects in the scene that would generate pupil-like regions in the image. One of the limitations of these systems is their use of thresholds when tracking the eyes. Defining thresholds can be difficult to define generically as the light conditions and head poses may influence the image observations of the eye. In this paper we propose a method based on particle filter- ing for tracking the eye. The method uses a new likelihood model for the image observations which avoids explicit def- inition of features and corresponding thresholds. For initial detection a method based on a cascaded classifier is used. The proposed method uses an near infrared illumination (780 nm) to produce the bright pupil effect. The method is illustrated in figure 1 and consists of two parts: eye detec- tion and eye tracking. Both methods are accomplished by simultaneously using the bright/dark pupil effects under ac- tive IR illumination and statistics of the eye appearance pat- tern under ambient illumination. The eye detection method