Head movements in context of speech during stress induction Giorgos Giannakakis * , Dimitris Manousos * , Panagiotis Simos †* , and Manolis Tsiknakis ‡* * Institute of Computer Science, Foundation for Research and Technology Hellas (FORTH), Heraklio, Greece Email: ggian@ics.forth.gr † School of Medicine, Division of Psychiatry, Heraklio, Greece ‡ Department of Informatics Engineering, Technological Educational Institute of Crete, Heraklio, Crete Abstract—This paper focuses on the analysis of head move- ments in the context of speech during stress and neutral conditions. An experimental protocol was followed including tasks of neutral and stressful emotional states induced by different types of stressors. Translational and rotational head movements and velocities were computationally estimated from 2-dimensional facial landmarks in order to assess stress states. In parallel, the effect of speech production on head motility was investigated. The results indicate that stress conditions increase head motility, in both translational or rotational movement features. Besides, there is a clear involvement of speech in the increase of head motility. However, the intensity of head motility can be attributed to the combined effect of stress or arousal and speech and not just the effect of speech production. Keywords-stress; head movements; head pose; speech; AAM I. I NTRODUCTION Head movement behaviour is considered to be part of non- verbal communication that is expressed in various aspects of everyday life. They belong to the human’s upper part body gestures serving as signs in the course of human interaction as well as of emotional states in general. There are several head actions such as lowering, raising, tilting, nodding, shak- ing which have specific meanings and they are recognizable in cross-cultural communication. It is interesting that head gestures are observed also in blind persons that share many, if not all, characteristics of similar behaviour among persons with intact vision [1], [2]. Alongside their communicative importance, head move- ments may convey affective information. In [3], head motion was modelled in order to recognize behavioural charac- teristics, among them expression of positive and negative emotions, while in [4] head movements presented signif- icant differentiations between positive and negative affect in infants. In [5], gesture analysis performed using among other features head motion information in order to recognize affective states with increased accuracy. In [6] head nods and shakes were employed among other features in order to discriminate complex emotional situations. Head movements related features have been used in limited studies estimating whether a subject is in a stressed state. It has been reported that head movements during stressful conditions are more frequent [7], more rapid [8] and that, in general, there is greater overall head motion [9], [10]. In addition, there are studies arguing that head movements are modulated by ongoing speech [11]–[13]. In [13], the linguistic functions of head movements as well as their se- mantics during conversations are presented. Moreover, head shifts were associated with the text structure and prosody [14]. Also, head nodding occurs when a person wants to emphasize a section of his/her speech or a specific word. In [15], head postural shifts (PS) are defined as wide, linear movements which occur mainly before speech initiation. In the literature, studies addressing the problem of stress detection through head motion features are limited. More- over, to our knowledge there is no combined analysis of stress and ongoing speech on head movements. This study investigates affective information both the independent as well as the combined effect of stress and overt speech on head movement patterns. II. METHODS A. Experimental procedure and population Data were recorded during the first data acquisition cam- paign (Semeoticons Reference Dataset SRD’14) within the context of the research project described in [16]. From this dataset, the recordings regarding stress assessment (Semeoti- cons Reference Dataset for Stress Assessment SRDSA’14) were used which are described in detail in [10]. The study sample included 23 participants (16 males, 7 females) aged 45.1±10.6 years. Each participant performed the whole experimental procedure (all the 12 different tasks) which means 69 neutral and 184 stressful sessions (69 with speech, 115 without speech) that were used in the analysis. Table I EXPERIMENTAL TASKS INVESTIGATED IN THE STUDY. # Experimental task Speech Duration (min) 1 Neutral (NT - 3 tasks) N 1 2 Interview (INT) Y 1 3 Reading Text (RT) Y 1 /2 4 Anxious event recall (AER) N 1 /2 5 Stressful event recall (SER) N 1 /2 6 Images from IAPS database (IAPS) N 2 7 Stroop Colour-Word Test (SCWT) Y 2 8 Adventure video (AV) N 2 9 Psychological pressure video (PPV) N 2