The Second International Conference on Music Communication Science, 3-4 December 2009, Sydney, Australia http://marcs.uws.edu.au/links/ICoMusic09/index.html EXPLORING FUNCTION AND AESTHETICS IN SONIFICATIONS FOR ELITE SPORTS Nina Schaffert & Klaus Mattes Stephen Barrass A.O. Effenberg University of Hamburg University of Canberra Leibniz University Hannover Department of Human Movement Digital Design & Interactive Media Institute for Sport Science 20148 Hamburg, Germany ACT 2601, Australia 30167 Hannover, Germany ABSTRACT The potential for real-time sonifications to enhance attention for specific movement sections in athletics has led to a growing interest from the elite sports community. Early examples of sonifications for swimming and (ergometer) rowing have demonstrated the functionality of this idea. These pioneering sport sonifications use simple sine-tones to achieve real-time responsiveness. However further improvements in technology will enable more complex sounds to be synthesised, opening up the design possibilities. During the COST-SID workshop on sonification, different design possibilities were explored using the rowing movement and data as a case study. The results identify the potential to include the motivational and physiological aspects of music and the richness of natural sounds. In further related work the initial sine-wave sonification was trialed in action with rowers on the water with improvements in speed demonstrating the effectiveness. Responses to a questionnaire about the sound indicate that functionality is a primary dimension in the aesthetics of a sonification in elite sports. The results from the workshop combined with the questionnaire identify many issues and raise many questions about function and aesthetics in sonifications designed specifically for elite sports applications. 1. INTRODUCTION Michael Phelps (multiple gold medallist winner in Athens and Beijing) listens to music as part of his preparation for a race. "I like to get in my own world. When I'm getting ready for a meet, I always have headphones on, listening to rap music to get myself fired up". Athletes listen to music prior to competition to relax, metnall prepare [1], and to facilitate a state of “flow” [2]. Music has physiological effects on heart rate and adrenalin levels, and can be a stimulant [3]. Music can be motivating and extend endurance during an exercise session (for example during a marathon run) [4], diverting attention from fatigue and altering perceptions of exertion during workouts. Music is integral to the sequencing of routines and coordination of team activities in gymnastics, competitive dancing, and synchronised swimming. Movement and sound are naturally bound together [5] athletes often use these sounds as sources of information. For example a tennis player can anticipate the speed of the return volley from the sound of the opponent’s stroke. Elite rowers listen to the boat moving through the water to adjust their motion sequence, and there was a reduction in performance when they wore earplugs [6], [7]. Sound also animates movement in time to a beat. The high temporal resolution of human hearing [8] enables humans to hear very specific information about the timing of movements, and to synchronise movements with sounds [9]. The ears lead the eyes, and people respond to sudden sounds behind them by turning to see what happened. Humans listen all the time in order to navigate through the environment. The sounds of moving things and kinetic events are a “natural music” that could be thought of as “the soundtrack of everyday life”. Technological developments have enabled simple sounds to be synthesised in real-time on mobile devices. This in turn has allowed information from sensors such as accelerometers, GPS, force, flex, heart-beat and breathing to be sonified in real-time. The potential to use these sonifications to enhance attention in athletics has led to a growing interest from the elite sports community. Early examples of interactive sonifications for swimming and (ergometer) rowing have demonstrated the effectiveness of this idea [5]. Recent advances in audio technology make it now possible to synthesise more complex sounds on mobile devices, opening up a wide space of possible sonification designs. This paper describes a workshop in which eight researchers produced different sonifications of data from a study with elite rowers. The sonifications were not constrained by the current technological limits of mobile devices, and serve to explore designs that could be deployed on mobile platforms in the future. The following section provides background on the state of the art in the sonification of rowing. The following sections describe the sonification workshop and the sonifications that were produced. The discussion compares and contrasts the different sonifications, and identifies different theories and approaches. This analysis lays a foundation for furthering the theory and method of sonification design with a particular focus on applications in elite sports. 2. BACKGROUND A pilot study was conducted with elite junior athletes (N=8) and their coaches. In the first part of the study the athletes and coaches were shown a video of a training run synchronized with the sonified sound sequence. The sonification was produced from kinematic parameters (propulsive boat acceleration and horizontal boat velocity) measured during training sessions. An acceleration sensor and global positioning system (GPS) were used to measure the movement of the boat at different stroke rates. The acceleration traces were converted into auditory information using a measurement and feedback system called Sofirow (BeSB GmbH Berlin) [7], [10]. This system maps numerical