Automated vs Human Recognition of Emotional Facial Expressions of High-Functioning Children with Autism in a Diagnostic-Technological Context: Explorations via a Bottom-Up Approach Miklos Gyori 1,2(&) , Zsófia Borsos 1,2 , Krisztina Stefanik 1,2 , Zoltán Jakab 1 , Fanni Varga 1 , and Judit Csákvári 1 1 ELTE University, Ecseri Road 3, Budapest 1097, Hungary gyorimiklos@elte.hu 2 MTA-ELTE ‘Autism in Education’ Research Group, Ecseri Road 3, Budapest 1097, Hungary maszk@barczi.elte.hu Abstract. Early detection of autism spectrum conditions (ASC) is an important goal. Automated facial expression recognition is a promising approach and has implications for assistive and educational technologies, too. This study was an initial exploration of (1) the inter-rater reliability of human recognition of facial emotions of high functioning (HF) children with ASC; (2) the relationship between human and automated recognition of facial emotions; and (3) a ‘bottom-up’ approach on identifying ASC/typical development (TD) differ- ences, from a screening serious game context. Thirteen HF, kindergarten-age children with ASC and 13 children with TD, matched along age and IQ, par- ticipated. Emotion recognition was administered on video-recordings from sessions of their playing with the serious game. Results showed lack of inter-rater reliability in human coding, confirming some advantages of machine coding. The simple bottom-up cross-sectional exploratory analysis did not reveal any ASC/TD difference. This is in contrast with our and others’ previous results, indicating such differences when aggregating emotion data from wider time-windows in machine-coded data-sets. This suggests that this second approach may be a more promising one to identify autism-specific emotion expression patterns. Keywords: Autism spectrum conditions Á Emotional facial expressions Screening Á Serious game 1 Background 1.1 Autism Spectrum Conditions, Technology-Aided Early Recognition Autism spectrum conditions (ASC) are atypical pathways of neurocognitive develop- ment, manifested in atypical patterns of social and communication skills, and the overall organization of own behaviors and interests [1, 2]. Resulting adaptation dif ficulties © Springer International Publishing AG, part of Springer Nature 2018 K. Miesenberger and G. Kouroupetroglou (Eds.): ICCHP 2018, LNCS 10896, pp. 466–473, 2018. https://doi.org/10.1007/978-3-319-94277-3_72