Journal on Multimodal User Interfaces manuscript No. (will be inserted by the editor) A Dataset for Point of Gaze Detection using Head Poses and Eye Images Christopher D. McMurrough · Vangelis Metsis · Dimitrios Kosmopoulos · Ilias Maglogiannis · Fillia Makedon Received: date / Accepted: date Abstract This paper presents a new, publicly avail- able dataset 1 , aimed to be used as a benchmark for Point of Gaze (PoG) detection algorithms. The dataset consists of two modalities that can be combined for PoG definition: (a) a set of videos recording the eye motion of human participants as they were looking at, or follow- ing, a set of predefined points of interest on a computer visual display unit (b) a sequence of 3D head poses syn- chronized with the video. The eye motion was recorded using a Mobile Eye-XG, head mounted, infrared monoc- ular camera and the head position by using a set of Vicon motion capture cameras. The ground truth of the point of gaze and head location and direction in the three-dimensional space are provided together with This work is supported in part by the National Science Foun- dation under award numbers CNS 0923494 and CNS 1035913. Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. Christopher D. McMurrough (Corresponding author) The University of Texas at Arlington E-mail: mcmurrough@uta.edu Vangelis Metsis The University of Texas at Arlington E-mail: vmetsis@uta.edu Dimitrios Kosmopoulos Rutgers, The State University of New Jersey E-mail: dk598@cs.rutgers.edu Ilias Maglogiannis University of Central Greece E-mail: imaglo@ucg.gr Fillia Makedon The University of Texas at Arlington E-mail: makedon@uta.edu 1 The dataset can be downloaded from: heracleia.uta.edu/eyetracking the data. The ground truth regarding the point of gaze is known in advance since the participants are always looking at predefined targets on a monitor. Keywords gaze tracking · head tracking · point of gaze · dataset 1 Introduction Eye tracking is a research problem of great interest, due to its large array of applications ranging from medical research, to human-computer interaction and market- ing research. In the era of ubiquitous and mobile com- puting, the utilization of eye tracking modules enables the development of easy to use communication inter- faces with assistive devices and systems. In most ap- plications, accurate detection or tracking of the Point of Gaze (PoG) is a prerequisite for the development of successful systems to utilize input via eye motion. Var- ious researchers have investigated the problem of gaze tracking and PoG detection and have proposed meth- ods to deal with it [2, 4, 6, 7, 9–12, 16]. Authors in [2] present one of the first computer vi- sion based methodologies proposed for non-intrusive de- tection of the position of a user’s gaze from the ap- pearance of the user’s eye. A three-layer feed forward- network, trained with standard error back propagation, is used for this purpose, while an accuracy of 1.5 degrees is reported. A real-time stereo-vision face tracking and gaze detection system is presented in [9]. In [12] authors describe the FreeGaze tracking system, which detects gaze position by the pupil and the Purkinje images. The gaze position is computed from these two images by using an eyeball model, specific for each user. Then, a personal calibration is proposed in order to achieve accurate gaze direction estimation. In [16] the cornea