QUANTITATIVE ANALYSIS OF THE IMPACT OF MIXING ON PERCEIVED EMOTION OF SOUND- SCAPE RECORDINGS Jianyu Fan Miles Thorogood Kıvanç Tatar Philippe Pasquier Simon Fraser University jianyuf@sfu.ca Simon Fraser University mthorogo@sfu.ca Simon Fraser University ktatar@sfu.ca Simon Fraser University pasquier@sfu.ca ABSTRACT Sound designers routinely mix source soundscape record- ings. Previous studies have shown that people agree with each other on the perceived valence and arousal for soundscape recordings. This study investigates whether we can compute the perceived emotion of the mixed- soundscape recordings based on the perceived emotion of source soundscape recordings. We discovered quantifia- ble trends in the effect of mixing on the perceived emo- tion of soundscape recordings. Regression analysis based on the trajectory observation resulted in coefficients with high R 2 values. We found that the change of loudness of a source soundscape recording had an influence on its weight on the perceived emotion of mixed-soundscape recordings. Our visual analysis of the center of mass data plots found the specific patterns of the perceived emotion of the source soundscape recordings that belong to differ- ent soundscape categories and the perceived emotion of the mix. We also found that when the difference in va- lence/arousal between two source soundscape recordings is larger than a given threshold, it is highly likely that the valence/arousal of the mix is in between the va- lence/arousal of two source soundscape recordings. 1. INTRODUCTION Audio-based creative practices, such as sound design and soundscape composition, often use recordings to create musical works. A soundscape recording (or field record- ing) is “a recording of sounds at a given locale at a given time, obtained with one or more fixed or moving micro- phones” [1]. Often, sound designers select source sound- scape recordings and carefully mix them together, which has a profound influence on meaning, significance, and perceived emotion. Together, the mixed-soundscape re- cordings create a rich, cohesive experience. Previous studies demonstrate that people have a high level of agreement on the perceived emotion of source soundscapes recording [2]. It is also possible to build machine-learning models to predict the perceived emo- tion of soundscape recordings [3]. However, to our knowledge, no study has been presented regarding the effect of mixing on the perceived emotion of soundscape recordings. In this study, we focus on the effect of mixing on the perceived emotion of soundscape recordings. We used Emo-Soundscapes, a dataset for soundscape emotion recognition that contains a group of annotated source soundscape recordings and annotated mixed-soundscape recordings [4]. The source soundscape recordings are selected following Schafer’s taxonomy so as to cover the diversity of soundscapes as much as possible [5]. The perceived emotion is represented as the ranking of a two- dimensional vector of valence and arousal [6]. As identi- fied by Thorogood and Pasquier [3], valence represents the pleasantness of a stimulus, which is used to report the perceived pleasantness of a soundscape recording. Arousal indicates the level of eventfulness. Next, we convert the annotators’ rankings to ratings and used regression models to determine the effect of mixing on the perceived emotion of soundscape record- ings. Moreover, we analyzed the center of mass data plots to find the relationships between the perceived emotion of the mixed-soundscape recordings and perceived emo- tion of source soundscape recordings that are selected within Schafer’s category. Last, we analyzed the likeli- hood of the perceived emotion of mixed-soundscape re- cordings lying between the perceived emotions of the two source soundscape recordings that are used for the mix. 2. RELATED WORKS 2.1 Taxonomy of Emotion and Affect Models Emotional responses are subjective with people having possibly a different response to the same stimulus. Ac- cording to previous studies [7], two types of emotions are involved when listening to soundscapes: • Perceived emotion: emotions that are communi- cated and expressed by the source. • Induced emotion: emotional reactions that the source provokes in an audience; it is what the audi- ence feels from the source. The perceived emotion is the emotion a source expresses. For example, the perceived emotion of happy songs is always “happy”. However, the induced emotion is more subjective. The same happy music may not necessarily induce happiness because of the internal interpretations and experiences of a listener. In this study, we focus on the perceived emotion of soundscapes. Copyright: © 2018 Fan et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License 3.0 Unport- ed, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.