A Digital Mobile Choir: Joining Two Interfaces towards Composing and Performing Collaborative Mobile Music Nicolas d’Alessandro numediart – Institute for New Media Art Technology of the University of Mons B-7000, Mons, Belgium nicolas@dalessandro.be Aura Pon Interactions Lab, Dept. of Computer Science of the University of Calgary T2N 1N4, Calgary, Canada aapon@ucalgary.ca Johnty Wang MAGIC – Media and Graphics Interdisciplinary Centre of the University of British Columbia V6T 1Z4, Vancouver, Canada johnty@ece.ubc.ca David Eagle Department of Music, Faculty of Fine Arts of the University of Calgary T2N 1N4, Calgary, Canada eagle@ucalgary.ca Ehud Sharlin Interactions Lab, Dept. of Computer Science of the University of Calgary T2N 1N4, Calgary, Canada ehud@cpsc.ucalgary.ca Sidney Fels MAGIC – Media and Graphics Interdisciplinary Centre of the University of British Columbia V6T 1Z4, Vancouver, Canada ssfels@ece.ubc.ca ABSTRACT We present the integration of two musical interfaces into a new music-making system that seeks to capture the expe- rience of a choir and bring it into the mobile space. This system relies on three pervasive technologies that each sup- port a different part of the musical experience. First, the mobile device application for performing with an artificial voice, called ChoirMob. Then, a central composing and con- ducting application running on a local interactive display, called Vuzik. Finally, a network protocol to synchronize the two. ChoirMob musicians can perform music together at any location where they can connect to a Vuzik central conducting device displaying a composed piece of music. We explored this system by creating a chamber choir of Choir- Mob performers, consisting of both experienced musicians and novices, that performed in rehearsals and live concert scenarios with music composed using the Vuzik interface. Keywords singing synthesis, mobile music, interactive display, inter- face design, OSC, ChoirMob, Vuzik, social music, choir 1. INTRODUCTION Over the last five years, the handheld form factor (i.e. smart- phone, portable gaming platform, e-reader, etc.) has grown in potential by several orders of magnitude in various as- pects. Particularly, local computational power, human in- teraction sensing capabilities, and connectivity have made a remarkable leap forward, leading to devices with a broader awareness of their overall context, including user intent, lo- cation, and social surrounding [7]. This progress in available resources brings handheld devices to take a dominant role in the emergence of digital media ecosystems, encompassing other types of form factors such as tablets, large interactive displays, tabletops, or wearable technologies. Therefore the concept of mobility is now ascribing to a broader defini- Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. NIME’12, May 21 – 23, 2012, University of Michigan, Ann Arbor. Copyright remains with the author(s). tion than its “pocket device” predecessor. It now refers to the pervasiveness of the computational power, towards the aim of creating an enriching medium of communication and expression, wherever and whenever the user desires it. The potential of this facet of mobile technology yields ex- citing promise for musical applications. We can design for mobile devices to serve as interfaces for musical expression by taking advantage of their mobility, touch sensors, and accelerometers. This trend has been vastly explored in the last few years [7, 5]. But the power and pervasiveness of these digital technologies in our society also affords an un- precedented ability to design tools for collaborative musical experiences. Indeed, we can also consider designing for the complete musical experience that mobile technology affords, which is equally about the context in which music is made, i.e. actual participants, their social relationships, and their setting, as it is about the individual instrument. There is a growing number of initiatives looking for a more holistic de- sign of mobile-based collaborative instruments [14]. Since the emergence of electronic instruments, and particularly computer-based instruments, various questions have been brought to our attention regarding the design criteria to be used in the development of collaborative interfaces for musical experience. A review is given in [1]. When it comes to orienting the design of a collaborative music-making system, the choice of an underlying music ensemble paradigm can be helpful, at least as a starting point for addressing the social aspects of envisioned ex- perience. Existing computer-mediated collaborative music- making systems often – explicitly or implicitly – make clear connections with historical ensemble types. The choice of such a music ensemble paradigm has a direct impact on various properties of the collaboration, e.g. how or if the piece is composed and displayed, how or if the piece is con- ducted and performed, the presence and prominence of hi- erarchy and leadership, the amount of of control allocated to individual performers vs. leader and composer, etc. In this work, we have chosen the choir as the music ensemble paradigm to orient our design. The choir has a highly hier- archical structure. Singers mainly work with pre-composed material. They use codified vocal registers and standard vocal effects. We also highlight the strong leadership from the conductor, in both timing and dynamics. Based on the choir as our design choice, we introduce a new system for bringing the choir musical experience into