Borderlands: An Audiovisual Interface for Granular Synthesis Chris Carlson CCRMA - Stanford University 600 Lomita Dr. Stanford, CA 94305 carlsonc@ccrma.stanford.edu Ge Wang CCRMA – Stanford University 600 Lomita Dr. Stanford, CA 94305 ge@ccrma.stanford.edu ABSTRACT Borderlands is a new interface for composing and performing with granular synthesis. The software enables flexible, real- time improvisation and is designed to allow users to engage with sonic material on a fundamental level, breaking free of traditional paradigms for interaction with this technique. The user is envisioned as an organizer of sound, simultaneously assuming the roles of curator, performer, and listener. This paper places the software within the context of painterly interfaces and describes the user interaction design and synthesis methodology. Keywords Granular synthesis, painterly interfaces, improvisation, organized sound, NIME, CCRMA 1. MOTIVATION Max Mathews famously posited that any sound can be represented by a sequence of digits, with the consequence that, given a computer with enough power, the entire audible universe may be explored. In recent years, he was careful to clarify his original theorem, pointing out that most of the sounds that can be made by computers are, “uninteresting, horrible, or downright dangerous to (oneʼs) hearing!” [14]. Mathews emphasized the importance of seeking sounds that are beautiful, while recognizing that the perception of beauty is a subjective matter. This line of research goes hand in hand with John Cageʼs idea of the composer as an organizer of sound [3] or, as Brian Eno recently suggested, the metaphor of the composer-as-gardener [6]. Enoʼs metaphor likens the creation of music to the act of planting a variety of seeds, observing their growth and evolution, and iteratively pruning the results into a one of many potential forms. Given that the human mind is underequipped to imagine the entirety of the set of all beautiful sounds, let alone the incredibly vast audible universe, a composer who desires to break new sonic ground often engages in this iterative process. The act of listening is then essential to the generation and evolution of new ideas. The conceptual framework provided by Mathews, Cage, and Eno forms a foundation for Borderlands, which stems from the desire to strip away common musical paradigms such as notation, scales, tracks, and arrangements and allow users simply explore and transform sound. The software exists in a space in which composition, performance, visualization, and, Figure 1. Borderlands main interface with multiple grain clouds operating on an overlapping landscape of sounds. most importantly, listening, overlap. The user is envisioned as an organizer of sound, simultaneously assuming the roles of curator, performer, and listener. This work is closely related to a class of instruments described in detail by Golan Levin in his Master’s thesis, Painterly Interfaces for Audiovisual Performance. In his dissertation, Levin enumerates a set of key design goals for the ideal audiovisual interface [quoted from 11]: • The system makes possible the creation and performance of dynamic imagery and sound, simultaneously, in real- time. • The system’s results are inexhaustible and extremely variable, yet deeply plastic. • The system’s sonic and visual dimensions are commensurately malleable. • The system eschews the incorporation, to the greatest extent possible, of the arbitrary conventions and idioms of established visual languages, and instead permits the performer to create or superimpose her own. • The system’s basic principles of operation are easy to deduce, while, at the same time, sophisticated expressions are possible and mastery is elusive. Direct and immediate interaction, expressiveness, and flexibility both sonically and visually – these objectives are difficult to completely realize in every system, but, when considered holistically, the results can be compelling. Tarik Barri’s Versum [2], for example, is a hybrid tool in which users compose by building a virtual 3D universe of custom audiovisual objects and perform by flying through the space. In this software, graphics and sound are inextricably linked – the visual is not merely a representation of the sound, it is the sound. This system engenders a new compositional process in which the form of a piece is equally influenced by both sound Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. NIME’12, May 21-23, 2012, University of Michigan, Ann Arbor. Copyright remains with the author(s).