The relationship of psycho-physiological responses and self-reported emotions while listening to music Frederik Nagel a , Oliver Grewe a , Reinhard Kopiez b , Eckart Altenm¨ uller a a Institute for Music Physiology and Musicians’ Medicine b Institute for Research in Music Education Hannover University of Music and Drama, Germany Since Kate Hevner’s [1] early investigations on perceived emotions while listening to music in 1936, there have been many different approaches to the measurement of emotions. For example, Gabrielsson and Lindstr¨om [2] investigated expression in speech while Schubert [3] tried to measure emotional expression by use of a continuous response method in a 2-dimensional emotion space (2DES). Measurements of physiological responses while listening to music have been done by numerous researchers since the 1960s. They found that changes in skin conductivity response play a central role within the group of physiological parameters. We investigated the emotions of 35 subjects induced by pre-selected standard musical pieces and by their favourite pieces. The aim was to reveal whether musical universals based on specific psychoacoustic cues exist, eliciting inter-individual comparable emotions. In particular we were interested in strong emotions such as chills, which commonly are described as shivers or goose- pimples. We combined a continuous self report technique with psycho-physiological measures. The researcher-developed software ”EMu- Joy” was developed as an extension of Schubert’s method, which enabled subjects to express their emotions ”on-line” while listening to music. The dimensions used were valence (negative-positive) and arousal (high-low). Subjects controlled the software with a computer mouse. One of the mouse buttons was used to indicate chill experiences. As psycho-physiological measures, heart rate (HR), skin conductivity response (SCR) and level (SCL), skin temperature, breathing rate and an electromyogram of two facial muscles (the cygomaticus and the corrugator) were recorded. In contrast to previous studies, subjects were instructed to express their own perceived emotions and not to rate the emotional expression intended by the composer. The experiment started with a learning phase during which subjects got accustomed to the continuous rating procedure. Handling the ”EMuJoy” software was trained by rating perceived emotions while looking at ten pictures from the International Affective Picture System [4]. These pictures were selected according to the two-dimensional emotion space covering the edges and the center (arousing, calming, negative, positive valence, The subjects listened to 7 selected pieces, 5 pieces were purely instrumental music and 2 included voice. After each piece, they had to fill in questionnaires concerning their associations related to the respective piece of music. At the end of the experimental session, information on musical expertise and personality factors was collected. SCR and chills were positively correlated in all subjects. Additionally, there was a significant positive correlation between chills and increase in heart rate. We found no inter-individually consistent rules for the relationship between self report and physiological data. However, subject specific correlations between these parameters could be detected. Furthermore, there are smaller groups of subjects that show similarities in their reactions to music. As a common rule, strong changes in arousal or valence were linked to SCRs and increasing HR. By coupling SCR and HR we had an improved procedure to distinguish between non-specific SCRs and autonomous reactions to music. Chill reports and changes in both parameters were correlated significantly. All subjects had more chills when listening to their own favourite music as compared to the pre-selected standard pieces. According to the data, there is no simple stimulus-reaction pattern with respect to the relation between music and emotion. The emotional self monitoring as well as the physiological reactions are influenced by individual factors, such as musical expertise and experience, vigilance, etc. Although there are groups of individuals with similar reactions to music, only the self reported peak experiences are significantly correlated with the physiological data. This is probably due to the fact that the reported chills are based on sympathetic activations. In a further step, we will attempt to identify distinct inner states of subjects while listening to music and relate these to specific physiological reactions. This relationship might by very individual. However, as has been demonstrated in speech recognition, these individual features can be used for mapping physiological reactions to emotional states. As in speech recognition where the characteristics of speech can easily be correlated with words, the inner states can be correlated with physiological data (see [5]). The work was supported by the DFG (Al 269-6) and the Center for Systemic Neurosciences Hannover References [1] K. Hevner. Experimental studies of the elements of expression in music. American Journal of psychology, 48:246–286, 1936. [2] A. Gabrielsson and S. Lindstrm Wik. Strong experiences related to music: A descriptive system. Musicae Scientiae, 7(2):157–217, 2003. [3] E. Schubert. Measuring temporal emotional response to music using the two dimensional emotion space. In Proceedings of the 4th International Conference for Music Perception and Cognition, pages 263–268, Montreal, Canada, 1996. [4] Cuthbert BN Lang PJ, Bradley MM. International affective picture system (iaps): technical manual and affective ratings. 1999. [5] R. W. Picard, E. Vyzas, and J. Healey. Toward machine emotional intelligence: Analysis of affective physiological state. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001. Göttingen NWG Conference 2005 #105B