Projection-based head-mounted display with eye-tracking capabilities Costin Curatu a , Hong Hua b , and Jannick Rolland a a College of Optics and Photonics: CREOL/FPCE, University of Central Florida, Orlando, FL 32816 b Optical Sciences Center, University of Arizona, Tucson, AZ 85721 ccuratu@creol.ucf.edu ABSTRACT We propose a novel conceptual design for a Head-Mounted Projection Display (HMPD) with Eye-Tracking (ET) capabilities. We present a fully integrated system that is robust, easy to calibrate, inexpensive, and lightweight. The HMD-ET integration is performed from a low-level optical configuration in order to achieve a compact, comfortable, easy-to-use system. The idea behind the full integration consists of sharing the optical path between the HMD and the Eye-Tracker. Along with lens design and optimization, system level issues such as eye illumination options, hardware alternatives are discussed. Keywords: Head-mounted display, Eye-tracking, Wearable displays, Displays 1. INTRODUCTION While head-mounted display (HMD) technologies have undergone significant developments in the last decade, they have suffered from tradeoffs and limitations in capability, which impose critical effects on visualization accuracy, user performance. Among the tradeoffs and limitations, the ignorance of eye movement is often an overlooked aspect. The functional benefits of an integrated HMPD-ET solution for human-computer, multi-modal interfaces, and gaze-contingent foveated displays have been recognized, but only very few and preliminary efforts have been made towards a low-level integration. The objective is to optimize the conceptual design of the HMD-Eye-tracker integration from a low-level optical configuration rather than to integrate functionality by adding up commercially available displays and eye-trackers 1,2 . We expect that a low-level integration will significantly improve the performance of both eye-tracking accuracy and display quality as it relates to accuracy and precision of registration of real and virtual objects in augmented environments. Such a system could have a wide range of applications in different fields of science and technology. Eye- tracking capability could be used to design a fovea-contingent display 3,4 . Another application could be a novel interactive interface for people with proprioceptive disabilities, where eye gaze instead of hands or feet can be used as a method of interaction and communication. Furthermore, eye-tracking capability in HMDs can provide more accurate eye-movement monitoring devices for human factors and vision research. Finally, eye-tracking capability in HMDs can be used as a metric to assess behavior in virtual environments in order to quantify the effectiveness of the technology in various specific tasks including training , education, and augmented cognition tasks. In section 2 of this paper we first review HMPD technology and how it differs from the more conventional HMD design. In section 3 we review current eye-tracking techniques and justify our choice of using the video oculography method. In section 4 we present the integration process, conceptual and optical designs, and the eye illumination scheme. Novel Optical Systems Design and Optimization VIII, edited by José M. Sasián, R. John Koshel, Richard C. Juergens, Proceedings of SPIE Vol. 5875 (SPIE, Bellingham, WA, 2005) · 0277-786X/05/$15 · doi: 10.1117/12.618715 Proc. of SPIE 58750J-1