Real Time Tracking of Musical Performances Antonio Camarena-Ibarrola ⋆ and Edgar Ch´ avez Universidad Michoacana de San Nicol´as de Hidalgo Morelia, Mich., M´ exico {camarena,elchavez}@umich.mx Abstract. Real time tracking of musical performances allows for imple- mentation of virtual teachers of musical instruments, automatic accom- panying of musicians or singers, and automatic adding of special effects in live presentations. State of the art approaches make a local alignment of the score (the target audio) and a musical performance, such procedure induce cumu- lative error since it assumes the rendition to be well tracked up to the current time. We propose searching for the k-nearest neighbors of the current audio segment among all audio segments of the score then use some heuristics to decide the current tracked position of the performance inside the score. We tested the method with 62 songs, some pop music but mostly clas- sical. For each song we have two performances, we use one of them as the score and the other one as the music to be tracked with excellent results. Keywords: entropy, index, proximity. 1 Motivation Real time tracking of musical performances consists in establishing the position of the current short segment of audio of a musical rendition of a test song as is being played in relation to the score (the target audio). Among the possible applications of real time tracking of musical performances we will briefly explain three of them: The virtual music teacher, Automatic accompanying of single players or singers, and automatic adding of special effects in live presentations. For the virtual music teacher, the score is a well played musical piece inter- preted by a trained musician. The music to be tracked is that generated by a student. Normally, a music teacher handles only one student at a time, as the student plays his instrument, the teacher gives him correction indications or gives his approval at the end. A virtual teacher should do the same, it must use the audio signal generated by the student, process it in real time and provide in- dications to the student accordingly. Of course, the great advantage of a virtual teacher is that it may multiply easily to manage many students simultaneously. ⋆ Corresponding author. G. Sidorov et al. (Eds.): MICAI 2010, Part II, LNAI 6438, pp. 138–148, 2010. c  Springer-Verlag Berlin Heidelberg 2010