Sense of Space: The Audience Participation Music Performance with High-Frequency Sound ID Masami Hirabayashi Institute of Advanced Media Arts and Sciences 4-1-7 Kagano Ogaki, Gifu, Japan hrr@iamas.ac.jp Kazuomi Eshima Laatry Creative Director Katsuratokudaijikitachou Kyoto city, Kyoto, Japan Kazuomi.e1981@gmail.com ABSTRACT We performed the musical work “Sense of Space” which uses the sound ID with high frequency sound DTMF. The IDs are embedded into the music, audiences' smartphones and tablets at the venue reacted to the IDs and then they play music pieces. We considered the possibility for novel music experiences brought about through the participation of audiences and spreading sound at the music venue. Keywords Sound ID, audience participation, music performance 1. INTRODUCTION According to the popularization of media technologies, systems that enhance music experiences with real-time and interactive technologies have been attracting attention. We make practical research for novel experiences and novel ways of enjoying music at music venues by introducing methodologies of expression in media- arts. In NxPC.Lab [6], our activity, we have made club-music events and developed systems and art-works to enhance music experiences. In particular we developed the system Cryptone [5] which enables interaction between performers and audiences by utilizing high- frequency sound – the so called "mosquito sound" – as a sound ID. The sound ID, which we refer to as Ultra Sound Communication (USC), has features that are easy to use for musicians, and easy to set up for music venues. We made a system that utilizes USC, and made the music performance "Sense of Space" in which audiences can participate, and produce the space/venue surrounded by music. In this paper, we describe the system and two live performances with different setups. 2. RELATED WORKS For interactive stage effects in music venues, the iPhone application SYNK [8] is a famous work. That was made for the world tour of Plastikman, where sounds and LED-lighting in the venue can be controlled by the audience with the application. Recently, wireless controlled light-pens [12] or Wrist-bands [13] are starting to be utilized at major pop concerts. For music artworks, the Sine Wave Orchestra [9] is similar to our concept. In that work, the audience has balls of sound that have particular frequency sinewaves, and generate music harmony of sinewaves when the audience gathers. It enables the participation of the audience for the music performance. SWARMED [1] is also a musical work where the audience can participate. In this work, the audience can control music through mobile devices connected with WiFi. In the area of sound IDs utilizing high-frequency sound, there is InfoSound [3], which was developed by YAMAHA, Sonic Notify [11], which is being used on YouTube, and so on. Automatic Contents Recognition technologies such as Evixar [2] or Intrasonics [4] can also be thought of as utilizing high-frequency sound. 3. SENSE OF SPACE The objective of the work "Sense of Space" is to achieve live music performances in which audiences can participate. We are conducting research that focuses on the interaction between performers and their audiences. It is generally difficult to join music performances with sounds made by the audience. In this work, music played by the performer sounds from the main PA system, with audience members being able to participate in the music with sounds played from their smartphones. The sounds from the PA and the smartphones make the music work as a whole and it spreads to the venue. This satiation brings a sense of unity for the performer, the audience and its venue, and also allows them to feel the atmosphere of other audiences and venue spaces with sounds played by smartphones at various locations. All participants can sense the other audiences and expanse of the venue, and because of this we have named this work “Sense of Space”. The quality of the music is the most important element for sufficient music experiences. Therefore we made music in cooperation with Kafuka (the artist name for Kazuomi Eshima), who created the composition while considering the synchronization of music pieces and the special property of venues. The music consist of the main melody and rhythm part sounds from the main PA and 16 music piece sounds from iPads placed in the venue and iPhones held by audience members. These music pieces start to play when triggered by the number of sound ID, which are associated to the music piece and are embedded in the music from the main PA. All of the music from the main PA and music pieces from iPad/iPhone are merged into the venue to compose the entire music work. The performer can flexibly play the music because the sound IDs can be embedded into the music in advance or generate/play real-time during the performance. The usage of sound IDs has the following merits compared to other methods (WiFi, Bluetooth etc.): - Easy to manipulate for musicians; embedded into the music, generate in real-time, etc. - Easy to compose synchronization of music; sound and sound IDs are transmitted at the same speed. These features can bring improvement to the quality of music when the artist composes music. 4. ULTRA SOUND COMMUNICATION (USC) We developed a method of communication by using DTMF (Dual Tone Multiple Frequency) with high frequency sound, which we call Ultra Sound Communication (USC). USC is the method of defining sound IDs by using the DTMF method with sounds from 18000Hz to 20000Hz. In general, humans can hear sounds from 20Hz to 20000Hz, but it is said that people over 20 years old cannot hear sounds over 16000Hz. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. NIME’15, May 31-June 3, 2015, Louisiana State University, Baton Rouge, LA. Copyright remains with the author(s). 58 Proceedings of the International Conference on New Interfaces for Musical Expression, Baton Rouge, LA, USA, May 31-June 3, 2015