- 1 - dots: an Audio Entertainment Installation using Visual and Spatial- based Interaction Nikolaos Grigoriou, Nikolaos Moustakas, Andreas Floros and Nikolaos Kanellopoulos Dept. of Audiovisual Arts, Ionian University, 49 100 Corfu, Greece {av200432, av200406, floros, kane}@ionio.gr Abstract. “dots” is an interactive sound installation that takes into account the spatial position of an arbitrary number of participants in order to algorithmically synthesize an audio stream in real-time. The installation core is a software application developed during this work, which employs advanced video and audio processing techniques in order to detect the exact participants’ positions and to weight-mix short audio granules. Audio mixing is performed using a virtual spatial gridding of the installation space in two dimensions. The synthesized audio stream reproduction is combined with a number of appropriately designed visual effects, which aim to enhance the participants’ comprehension and render the “dots” installation a high-quality interactive audiovisual platform. 1. Introduction The continuous growth and evolution of digital audiovisual technologies is providing the scientific framework to design and develop/demonstrate new ways of artistic expression [1]. The recently established high-definition video and multichannel audio formats and standards are becoming widely accepted by both the media-producers’ and the end-users’ market. Additionally, new terms and ideas originating from the general concept of interaction are nowadays frequently used to provide novel means of audio and visual production, where the audience is actively participating in the production process [2], [3]. As a consequence, a continuously growing number of new media artists have now the option to express their thoughts and feelings by developing and demonstrating their installations, using high quality audio/visual synthesis and playback techniques and equipment [4]. The above interactive installations can be also considered as advanced tools for creating and synthesizing in real-time high- quality audio/visual content. For example, in a typical case, the visual information obtained or processed from an interactive installation can provide feedback to a sound synthesis system for producing novel sound streams in real-time. The opposite approach can be alternatively employed for synthesizing complex visual content and environments using sound recorded signals as input. In any case, the user interaction is usually realized using an interaction algorithm that defines the rules under which selected synthesis parameters are varied. The variation amount is usually obtained from the installation environment, typically using a wide range of wired or wireless sensors, image capturing devices and video and/or audio recording equipment [5]. In this article we present the “dots” interactive platform, which aims to produce complex audio streams taking into account the movement and the instantaneous spatial positions of the participants. In order to provide a motion motivation to the participants, an additional visual component is produced and concurrently projected in real-time, which also depends on the participants positions and motion. Hence, a complex spatial, audio and visual interaction effect is achieved which apart from dynamically creating audio content can be also considered as an interactive game with colours and sounds. The visual content consists of a controlled number of dots with varying colours, providing the name “dots” to the overall installation platform. The “dots” interactive platform presentation is here mainly performed in technological terms, typically focusing on the platform architecture and some elementary design / algorithmic issues and concepts. Additionally, within the framework of this work, we investigated and evaluated the interaction functionality and the overall installation performance under real-world conditions during an audiovisual festival / exhibition. This evaluation process also allowed the collection and interpretation of some observations related to the behaviour and the means of interaction of the participating audience. The rest of the paper is organized as following: In Section 2, the “dots’ installation architecture is described in detail, focusing on both the core application and the installation room requirements and design. Next, a demonstration of the installation interactive features is presented in Section 3, followed by a brief analysis of the functional and behavioural observations made during an installation exhibition. Finally, Section 4 concludes this work and accents further interaction and audio/visual enhancements that may be integrated in the “dots” platform in the future. 2. Analytic installation description As mentioned previously, the “dots” platform developed during the present work is an interactive audio/video installation, an interactive game, combining space, time and audio/visual content. From the design point of view, it consists of (a) the core application, which is responsible for handling the basic input / output video and audio signals and for realizing all the signal processing necessary for recognizing the spatial placement of the participants as well as for appropriately mixing and finally producing the audio (and video) signal and (b) the installation space, where the participants are moving and interacting with the core application, producing the final, complex audiovisual output. Both core application and installation space subsystems are described in detail in the following two Sections. 2.1. The “dots” core application The application was developed and programmed with the open source tool “Processing” [6], which represents a powerful software sketchbook and professional production tool used in many fields of signal processing, science/technology and arts. A number of particular Processing libraries were additionally used for implementing the “dots” installation application (see Figure 1). For example, advanced image and signal processing 112