An XML-based Synchronization of Audio and Graphical Representations of Music Scores Adriano Baratè, Luca A. Ludovico Laboratorio di Informatica Musicale (LIM) Dipartimento di Informatica e Comunicazione (DICo) Università degli Studi di Milano Via Comelico, 39/41 – 20135 Milan - Italy {barate, ludovico}@dico.unimi.it Abstract This paper presents an overview on a future IEEE standard aimed at providing an overall description of music. This format, known as IEEE PAR1599, is based on the XML meta-language. Its purpose is taking into account the heterogeneous multimedia representations of music, such as audio tracks, video clips and graphical instances of score. The key characteristics of the format are the possibility to enjoy such heterogeneous contents in a synchronized way and the possibility to switch from a particular representation to another in real-time. After a short description of the particular XML language we have adopted, a case study will be presented: an application installed at the exhibition "Celeste Aida" held at Teatro alla Scala in November 2006 - January 2007. 1. Introduction This paper deals with a music application supporting content-based browsing, synchronization and indexing of images, video and audio. In the following, we will describe a software which implements an interactive and advanced enjoyment of music thanks to the underlying XML-based format. As regards the latter aspect, at LIM 1 a standard language for symbolic music description is under development. This language, known as MX, is a meta- representation for describing and processing music information within a multilayered environment, in order to achieve integration among structural, score, interpretative, and digital sound levels of representation. Furthermore, the proposed standard should integrate music representation with already defined and accepted common standards. The development of the MX format follows the 1 Laboratorio di Informatica Musicale, Dipartimento di Informatica e Comunicazione, Università degli Studi di Milano. guidelines of IEEE PAR1599. 2 This recommended practice deals with applications and representations of Symbolic Music Information using the XML language, as described in [1]. The proposed IEEE PAR1599 comes at a point in time when it is possible to draw from the experience of existing, previous efforts. After standardization process, MX will be applicable to any kind of software dealing with music information, e.g. score editing, OMR, music performance, musical databases, composition, and musicological applications. This paper constitutes a preview of the MX format, which is currently undergoing the balloting phase of the IEEE standardization process. Besides, this work provides an example about the class of MX-based applications aimed at the integration of audio and video contents in music context. MX has been treated in detail in many papers already published and cited among the references, so in the following we will describe only the key characteristics of the format necessary to understand the final case study. The application based on MX technology is an evolution of other software tools implemented for research [2] or demonstrative purposes [3]. Its purpose is playing MX files containing symbolic, audio and video contents in a synchronized way. This application, besides implementing the concept of a comprehensive description of music in all its multimedia forms, has some practical implications as well. First, it supports a direct, intuitive and immediate approach towards music notation and listening, even for untrained people. Besides, it can be applied to any music genre, from any historical period and culture. For these reasons, we think that the technologies here described are aimed not only at academic research and cultural exhibitions, but also at the market of portable devices and multimedia entertainment in general. Finally, we think that such application can provide also a useful didactic instrument. 2 Project Authorization Request 1599 (PAR1599), the IEEE formal document about the definition of a commonly acceptable musical application using the XML language.