Artificial Organs 29(8):658–664, Blackwell Publishing, Inc. © 2005 International Center for Artificial Organs and Transplantation 658 Blackwell Science, LtdOxford, UKAORArtificial Organs0160-564X2005 International Society for Artificial Organs298658664Original Article Image Processing for Cortical StimulatorsL.-X. BUFFONI ET AL. Received July 2004; revised March 2005. Address correspondence and reprint requests to Dr. Mohamad Sawan, PolySTIM Neurotechnologies Laboratory, Department of Electrical Engineering, Ecole Polytechnique de Montreal, C.P. 6079, Succ. Center-Ville, Montreal, Quebec H3C 3A7, Canada. E-mail: mohamad.sawan@polymtl.ca Image Processing Strategies Dedicated to Visual Cortical Stimulators: A Survey Louis-Xavier Buffoni, Jonathan Coulombe, and Mohamad Sawan PolySTIM Neurotechnologies Laboratory, Department of Electrical Engineering, Ecole Polytechnique de Montreal, Montreal, Quebec, Canada Abstract: Multi-electrode devices are constantly evolving toward a state where complexity and reliability are ade- quate for providing a breakthrough in visual cortical stim- ulation allowing the blind to recover partial vision. Yet few research teams have focused on the development of the front-end subsystem that transforms an input image from a camera into stimulation commands. This article collects state-of-the-art knowledge about the appearance and organization of phosphenes, and previous work in image processing dedicated to visual cortical stimulation. Obser- vations and hypothesis about important issues are high- lighted, and six image processing strategies that could be used in such a subsystem are presented, from the most optimistic that use brightness modulation to emulate gray- scale to the most conservative that use only on/off pho- sphene evocation. Key Words: Phosphene—Phosphene pattern recognition—Visual cortical stimulation—Intrac- ortical microelectrodes—Scene representation—Image processing—Resolution reduction—Image segmentation. Since the early discovery that visual sensations can be evoked using direct electrical stimulation on top of the visual cortex (1), researchers have been work- ing to build a visual cortical stimulator (VCS) that would help the blind to recover partial vision. Studies in interrelated topics have been conducted in order to develop such VCSs. They can be classified into the following categories: study of physiological responses to electrostimulation, development of mic- roelectrodes, implementation of low-power micro- chips to be used in implanted devices, and evaluation of low-resolution image recognition with sighted vol- unteers using simulated prosthetic vision. Figure 1 illustrates a typical VCS. In such a system, a camera grabs images that are processed and trans- formed into stimulation commands to be sent to the implant via an inductive link. The implanted device is to be tested first in order to obtain reliable phos- phene evocation, and therefore allows multiple stimulation strategies (2). The camera and image pro- cessor constitute the front-end subsystem, which will then be used to study and/or validate the trans- formation of input device data into stimulation commands. The purpose of this subsystem is to represent scenes in the real world as meaningful patterns of phosphenes. To date, few significant advances have been made in the field of phosphene pattern recog- nition. It is often conceived that grayscale images can be directly generated in a patient’s mind by evoking phosphenes whose brightness matches the corre- sponding image pixels. However, results from bio- medical studies do not encourage such optimism (3). Therefore, we suggest that a system to be used for early in vivo testing should implement several image processing strategies in order to validate phosphene evocation and vision restoration. The strategies pre- sented later in this article are inspired by the descrip- tion of phosphenes given by human volunteers in previous studies. Reliable stimulation techniques and a large number of implanted electrodes are assumed. Image processing is hereby envisioned as an aid for mobility, in which a normal video camera is used as an input device. A physiological description of the phosphenes and their organization will be given in the second section