Design and Development of a Low-Cost Eye Tracking System for the Rehabilitation of the Completely Locked-In Patient Ziad O. Abu-Faraj, Ph.D., Senior Member, IEEE; Maya J. Mashaalany, Student Member, IEEE; Habib C. Bou Sleiman, Student Member, IEEE; Jean-Louis D. Heneine, Student Member, IEEE; and Waleed M. Al Katergi, Student Member, IEEE Department of Biomedical Engineering, American University of Science and Technology, Beirut, Lebanon Abstract – This article describes the design and development process of an eye tracking-based computer system that benefits from the intact ocular motor control of the completely locked-in patient to provide him or her with an alternative means of communication. A completely locked-in patient is an individual who has lost all types of motor control and communication ability with people in his or her environment. This developed system uses a head-mounted web camera to capture real-time images of the patient’s eye. These images are then passed to a program, developed using Matlab ® , which processes them and computes the coordinates of the pupil position. The program then sends commands to an interactive JAVA™-based interface, which provides the patient with a matrix of pictograms representing the most essential daily communication activities. When a pictogram is activated (clicked), the system plays back an audible statement, recorded in any language, reflecting the desired activity. Ten healthy adult volunteers, free from any musculoskeletal or neurological disorders, participated in the validation of the system. Validation results revealed a system accuracy of 96.11 5.58 % and repeatability of 94.44 2.51 %. The rehabilitative system developed in this project offers the locked-in patient, of any social class, the ability of simple yet effective communication. The advantages of this system over existing systems are low cost, low processing power, ease of operation, little training requirements, minimal disturbance to the patient, and ease of customization to any mother tongue. Keywords – Locked-in patient, brain-computer interface, eye tracking, rehabilitation engineering. I. INTRODUCTION In certain classes of neurological disorders where the central nervous system has been subjected to severe irreversible damage, conventional methods of treatment often turn out to be ineffective. Such disorders include, but are not limited to, cerebral palsy, cerebral aneurysm, traumatic brain injury, stroke, apraxia, and aphasia. In some severe cases, more than one disorder might strike the same patient and make him or her completely locked-in; i.e., cognitively intact, but unable to move or communicate [1]. Globally, there are regions where the completely locked-in patients are doomed to social rejection, specifically when conventional rehabilitation methods fail to alleviate the severity of the handicaps in these disabled individuals. For instance, in the absence of motor control, it is virtually impossible to use sign language or common input devices to interface with a computer system in order to communicate through spelling or expression-building software [2]. Brain-Computer Interface (BCI) methods for the rehabilitation of patients with neural disorders, similar to those previously enumerated, have been extensively reported in the literature. BCI systems are mainly categorized according to the utilized input technique. Commonly reported techniques include electromyography (EMG), electroencephalography (EEG), electrooculography (EOG) [3], eye tracking [2][4-9], and other custom designed input devices [10-11]. The decision about which input technique to be used depends on the specific case being studied [1]. In order to be functionally effective, the use of EMG-based BCI requires that the patient be able to control at least one- to-three distinct muscles. While, the head-mounted tilt sensor method [10] can be used only if the patient has the volitional ability of rotating his or her head. Accordingly, these two methods are precluded from consideration since locked-in patients do not possess any discernible movements or EMG activity. Subsequently, BCI methods employing eye tracking, EEG, or EOG remain feasible for applications involving locked-in patients [1]. Nevertheless, EEG systems are expensive, sophisticated, cumbersome, and require extensive training; while EOG data suffer from a lack of accuracy at the extreme position of the pupil; particularly, small angle displacements (less than 2 ) are difficult to record whereas large eye movements (greater than 30 ) do not produce bioelectric amplitudes that are strictly proportional to eye position [12]. As a result, eye-tracking systems would be more adequate for such applications and are easier to setup. This article describes the design and development of a low- cost eye tracking-based brain-computer interface system for the rehabilitation of the completely locked-in patient having an intact ocular motor control to serve as an alternative means of communication. The developed system has been designed according to the following specifications: low cost, low processing power, ease of operation, little training Manuscript received April 24, 2006 and accepted June 20, 2006. This work was supported in part by funds from the Research Council of the American University of Science & Technology (AUST, Beirut, Lebanon). Z. O. Abu-Faraj is the Chair of the Department of Biomedical Engineering at the American University of Science and Technology, Beirut, Lebanon (telephone: +961-1-218716 ext. 249; facsimile: +961-1- 339302; e-mail: bme@aust.edu.lb ), while the co-authors are senior-year biomedical engineering students at the same institution. Proceedings of the 28th IEEE EMBS Annual International Conference New York City, USA, Aug 30-Sept 3, 2006 SaBP4.4 1-4244-0033-3/06/$20.00 ©2006 IEEE. 4905