2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII) Computational Modeling of Psycho-physiological Arousal and Social Initiation of children with Autism in Interventions through Full-Body Interaction Batuhan Sayis, Ciera Crowell, Juan Benitez, Rafael Ramirez, Narcis Pares Department of Information and Communication Technologies Universitat Pompeu Fabra Barcelona, Spain [batuhan.sayis, ciera.crowell, juanpedro.benitez, rafael.ramirez, narcis.pares]@upf.edu Abstract—This study is part of a larger project that wants to foster social initiation behaviors in children with Autism Spectrum Disorder (ASD). We approach this through a full- body interactive Mixed Reality (MR) experience that mediates a face-to-face play session between an ASD child and a non-ASD child. The goal of this study is to obtain a data model that allows us to evaluate the goodness of the MR system compared to a typical social intervention strategy based on construction tools (in this case LEGO bricks) which acts as the control condition. In this paper we present our first analysis of the arousal generated by the MR experience compared to that generated in the control condition. We address this by analyzing psycho- physiological data recorded during the social interaction behaviors in the ASD child while playing with the non-ASD child. We followed a repeated-measures design with two conditions: our full-body interaction MR environment and the typical social intervention strategy based on LEGO bricks. To measure physiology, Electrocardiogram (ECG), Electrodermal Activity (EDA) and Accelerometer (ACC) data were acquired through a wearable designed by our lab. We used machine learning techniques to analyze the huge amount of multimodal data from the ASD children obtained during 18 trials (3 female and 15 male). As a result, we were capable of classifying social initiation behaviors of ASD children during the MR environment sessions and those occurring during the LEGO construction sessions based on the psycho-physiological data sources. This is a first sign showing that our MR system has specific properties, compared to a traditional construction- based intervention, which potentially provide a new interesting context to intervention in ASD. Keywords—Autism Spectrum Condition, Mixed Reality, Embodied Interaction, Social Initiation, Arousal, Psycho- physiology, Computational Modeling I. INTRODUCTION Although children with Autism Spectrum Disorder (ASD) can learn to respond to social initiations started by others, they may present major difficulties in initiating these social interactions by themselves [1]. One factor that may impede the ability to initialize social interactions might be anxiety. In Mixed Reality Environments, communication and interaction can extend beyond verbal exchanges to include embodied interaction, which contributes to social perception and social understanding. Moreover, children with ASD have been found to have an affinity towards information and communication technologies (ICT) [2]. Practicing socialization in these environments can be a way to reduce anxiety while simultaneously training or acquiring behavioral patterns. Full- body virtual environments strive to place the body as the center of focus, incorporating the use of gestures and non- verbal language which are key to interpersonal communication. However, we must be careful with the fact that children with ASD commonly show fluctuations in affect that may emerge from sensorial challenges and motor disturbances that can affect posture, ability to speak, and facial expressions. Therefore, the stress level of children with ASD might increase and their overt behavior can be inconsistent with their internal affective state [3,4,5]. Psycho-physiology can help us better determine the children’s internal state and compare it with the coding of their overt behaviors. It has become widely accepted that skin and cardiac activity, act as physiological markers that specifically reflect the emotional and cognitive states of a person [6,7,8,9,10]. As much as, MR, embodied interaction and psycho-physiological computing are increasingly showing potential in ASD research, the data from these systems are multimodal in nature and complex to analyze. It requires establishing a model of the complex relationship between different types of data sources. A data driven approach, such as machine learning, is ideally suited for this task. In the present study, we generated a computational model that allows us to understand the relation between the children's arousal activity and the overt social initiation behaviors shown by them; which we coded from video recordings of the play sessions. II. BACKGROUND Social interaction entails communication exchanges between individuals or groups in the form of initiations and responses [11]. Several reports have shown that high functioning children with ASD are able to sustain social interactions once initiated. However, they have lower rates of initiating social sequences and play by themselves; i.e. that they start a communication act by their own will [12]. Anxiety could be a factor that could hinder the ability to initialize social interactions. Hirstein, in 2001, showed a possible exacerbation of hyperactivity when subjects This work has been funded by Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Program (MDM-2015-0502).