Acta Astronautica 60 (2007) 341 – 350 www.elsevier.com/locate/actaastro Monitoring of facial stress during space flight: Optical computer recognition combining discriminative and generative methods David F. Dinges a , ∗ , Sundara Venkataraman b , Eleanor L. McGlinchey a , Dimitris N. Metaxas b a Unit for Experimental Psychiatry, Department of Psychiatry, University of Pennsylvania School of Medicine, 423 Guardian Drive, 1013 Blockley Hall, Philadelphia, PA 19104-6021, USA b Department of Computer Science, Center for Computational Biomedicine, Imaging and Modeling, Rutgers University, 110 Frelinghuysen Road, Piscataway, NJ 08854-8019, USA Available online 17 October 2006 Abstract Astronauts are required to perform mission-critical tasks at a high level of functional capability throughout spaceflight. Stressors can compromise their ability to do so, making early objective detection of neurobehavioral problems in spaceflight a priority. Computer optical approaches offer a completely unobtrusive way to detect distress during critical operations in space flight. A methodology was developed and a study completed to determine whether optical computer recognition algorithms could be used to discriminate facial expressions during stress induced by performance demands. Stress recognition from a facial image sequence is a subject that has not received much attention although it is an important problem for many applications beyond space flight (security, human–computer interaction, etc.). This paper proposes a comprehensive method to detect stress from facial image sequences by using a model-based tracker. The image sequences were captured as subjects underwent a battery of psychological tests under high- and low-stress conditions. A cue integration-based tracking system accurately captured the rigid and non-rigid parameters of different parts of the face (eyebrows, lips). The labeled sequences were used to train the recognition system, which consisted of generative (hidden Markov model) and discriminative (support vector machine) parts that yield results superior to using either approach individually. The current optical algorithm methods performed at a 68% accuracy rate in an experimental study of 60 healthy adults undergoing periods of high-stress versus low-stress performance demands. Accuracy and practical feasibility of the technique is being improved further with automatic multi-resolution selection for the discretization of the mask, and automated face detection and mask initialization algorithms. © 2006 Elsevier Ltd. All rights reserved. Keywords: Optical computer recognition; Computer vision; Stress; Human face; Astronauts; Markov models 1. Introduction The measurement of facial expressions is a well stud- ied problem in the fields of anthropology, psychology ∗ Corresponding author. Tel.: +1 215 898 9949; fax: +1 215 573 6410. E-mail address: dinges@mail.med.upenn.edu (D.F. Dinges). 0094-5765/$ - see front matter © 2006 Elsevier Ltd. All rights reserved. doi:10.1016/j.actaastro.2006.09.003 [1,2] and more recently computer vision [3–9]. Facial expressions are an important factor in understanding the emotional state of humans and even more so in the case of humans performing critical tasks while under stress. Astronauts are required to perform mission-critical tasks at a high level of functional capability throughout space- flight. While they can be trained to cope with, and/or adapt to some stressors of spaceflight, stressful reactions