10th International Society for Music Information Retrieval Conference (ISMIR 2009) BODY MOVEMENT IN MUSIC INFORMATIONRETRIEVAL Rolf Inge Godøy FourMs Department of Musicology University of Oslo r.i.godoy@imv.uio.no Alexander Refsum Jensenius FourMs Department of Musicology University of Oslo a.r.jensenius@imv.uio.no ABSTRACT We can see many and strong links between music and hu- man body movement in musical performance, in dance, and in the variety of movements that people make in lis- tening situations. There is evidence that sensations of hu- man body movement are integral to music as such, and that sensations of movement are efficient carriers of infor- mation about style, genre, expression, and emotions. The challenge now in MIR is to develop means for the extrac- tion and representation of movement-inducing cues from musical sound, as well as to develop possibilities for using body movement as input to search and navigation inter- faces in MIR. 1. INTRODUCTION There are strong links between music and body movement: Performers produce sound through movements, and listen- ers very often move to music, as can be seen in dance and innumerable everyday listening situations. The links between music and body movement have been discussed since antiquity, but it is mostly in the last decade that we have seen more systematic research efforts on this topic within fields such as music technology, music performance, and music cognition [1–3]. Despite this rapidly growing research in various music-related fields, the idea of body movement as an integral and ubiquitous part of both per- formance and perception of music seems so far not to have had many consequences for music analysis, music theory, and music information retrieval. Based on a quick survey of papers from recent ISMIR conferences as well as on the overview in [4], the papers that directly or indirectly are concerned with body movement seem limited to a few on query by humming and tapping, as well as some on beat tracking and tempo induction. Also, a cross-check on Google Scholar showed that out of 4670 hits on MIR, 3730 included “audio”, 1990 “MIDI”, while only 21 included Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. c  2009 International Society for Music Information Retrieval. “body movement”. 1 It seems fair then to conclude that body movement has not been an important topic in MIR contexts. Based on our own and various international colleagues’ work of the past decade, we believe that body movement is not just something that incidentally co-occurs with mu- sic, but that body movement is integral to music as a phe- nomenon. We would go so far as to claim that our expe- rience of music is based on the combination of sound and movement sensations, hence that music is a fundamentally embodied phenomenon [5, 6]. With such an understanding of music, it also becomes clear that sensations of music- related body movements are in fact highly salient features of music, and should be considered alongside various sonic features, e.g. pitch, melody, harmony, and timbre. Explor- ing music-related body movement then becomes an urgent task also in relation to MIR, and in this paper we shall try to give an overview of the kinds of body movement that could be of interest in MIR and how they can be studied. Finally, we shall present some suggestions for how body movements could be used in interfaces for the search and retrieval of music information. 2. MUSIC-RELATED MOVEMENT It seems that listeners associate different kinds of body movement with the music they hear, or merely imagine. Here it can be useful to start by making the general distinc- tion between sound-producing and sound-accompanying movements. Although this distinction may not always be so clear-cut, sound-producing movements are those that contribute to the production of musical sound, and sound- accompanying movements are those that are made in re- sponse to the sound being heard [3]. Sound-producing movements may further be divided into excitatory movements such as hitting, bowing, blowing, and modulatory movements such as those for making a vi- brato or various timbral nuances. Associated with sound- producing movements we also have various types of sound- facilitating, expressive, and communicative movements, mean- ing movements that are not strictly speaking sound-producing but still play an important role in music performance. Sound- accompanying movements, on the other hand, are all kinds of movements that people may make to music such as in 1 Search conducted 21 April 2009 using Google Scholar in English, and with a syntax of “Music Information Retrieval” + “. . . ”. 45