THE PHYSIOLOGY OF FEAR AND SOUND: WORKING WITH BIOMETRICS TOWARD AUTOMATED EMOTION RECOGNITION IN ADAPTIVE GAMING SYSTEMS Tom A. Garner 1 and Mark N. Grimshaw 2 University of Aalborg School of Communication AALBORG, Denmark 1 0045 71 58 73 38 2 0045 99 40 91 00 ABSTRACT The potential value of a looping biometric feedback system as a key component of adaptive computer video games is significant. Psychophysiological measures are essential to the development of an automated emotion recognition program, capable of interpreting physiological data into models of affect and systematically altering the game environment in response. This article presents empirical data the analysis of which advocates electrodermal activity and electromyography as suitable physiological measures to work effectively within a computer video game-based biometric feedback loop, within which sound is the primary affective stimuli. KEYWORDS Psychophysiology, biofeedback, affective sound, adaptive gameplay 1. INTRODUCTION The overarching problem that motivates this study is the insufficient capacity of computer software (specifically recreational computer video games [CVG]) to respond to the affective state of the user. Prior research has stated that this limitation significantly damages usability between human and computer (Picard, 2000). From a CVG perspective, the absence of an affect recognition system can limit: the effectiveness of social/emotional communication between players and virtual characters, the game’s capacity to respond to undesirable gameplay experiences such as boredom or frustration, the opportunity for the system to build an affective user-profile to automatically customise game experiences, and also, the potential to communicate emotions to other live players over a network. In the broadest sense, psychophysiology refers to study of the relationships that exist between physiological and psychological processes. Despite being a relatively young research field, that Cacioppo et al. (2007) describes as ‘an old idea but a new science’, psychophysiology has branched into a wide range of applications and has integrated with various other disciplines including dermatology (Panconesi & Hautmann, 1996) and psychopathology (Fowles et al., 1981). Modern psychophysiology was envisioned in response to the physiology/psychology divide problem (that between the two they provide a comprehensive explanation of human behaviour yet remain distinctly separate fields of study). Psychophysiological data acquisition addresses several problems experienced when evaluating emotions via self-report, such as affect insensitivity and emotion regulation (Ohman & Soares, 1994). Research has documented circumstances in which the agendas of the individual facilitate regulation (suppression, enhancement, false presentation) of outward emotional expression, providing severe reliability concerns if relying entirely upon visual analysis and self-report to interpret emotional state (Jackson et al., 2000; Russell et al., 2003). Biometric data collection has the potential to circumvent this problem via measurement of emotional responses characteristically associated with the autonomic nervous system (ANS) and is significantly less susceptible to conscious manipulation (Cacioppo et al., 1992).