Eye Movements in Speech Technologies: an overview of current research Mattias Nilsson Department of linguistics and Philology, Uppsala University Box 635, SE-751 26 Uppsala, Sweden Graduate School of Language Technology (GSLT) Abstract We present a summary overview of recent work using eye movement data to improve speech technologies. We summarize the experimental psycholinguistic evidence mo- tivating these applications and provide an overview of a number of gaze-speech studies in the areas of multimodal human-computer interaction, synthesized speech evaluation and automatic speech recognition. 1 Introduction When listeners follow spoken instructions to manipulate real objects or objects in a visual display, their eye-movements to the objects are closely time-locked to the spoken words referring to those objects (Eberhard et al., 1995). In other words, listeners naturally make saccadic 1 eye movements to objects as they recognize the spoken words referring to them. For the last fifteen years this central observation in psycholinguistic research has provided a wealth of insights into the time course of spoken language processing. More recently, a growing number of researchers in speech technology and human-computer interaction has drawn on the experimental evidence and are now using eye tracking to address diverse issues such as dialog system design, synthesized speech evaluation and automatic speech recognition. Currently, however, there is no designated forum for research on the ways in which eye movements may inform speech technologies, and papers addressing these questions are spread out and often published in quite different journals. Hence it is decidedly hard to get a general overview of the problems addressed, the methods used and the Email address: mattias.nilsson@lingfil.uu.se (Mattias Nilsson). 1 Saccades are very rapid eye movements that transport the eyes from one fixation point to another.