10th International Society for Music Information Retrieval Conference (ISMIR 2009) EVALUATING AND ANALYSING DYNAMIC PLAYLIST GENERATION HEURISTICS USING RADIO LOGS AND FUZZY SET THEORY Klaas Bosteels Ghent University, Gent, Belgium klaas.bosteels@ugent.be Elias Pampalk Last.fm Ltd., London, UK elias@last.fm Etienne E. Kerre Ghent University, Gent, Belgium etienne.kerre@ugent.be ABSTRACT In this paper, we analyse and evaluate several heuristics for adding songs to a dynamically generated playlist. We explain how radio logs can be used for evaluating such heuristics, and show that formalizing the heuristics using fuzzy set theory simplifies the analysis. More concretely, we verify previous results by means of a large scale eval- uation based on 1.26 million listening patterns extracted from radio logs, and explain why some heuristics perform better than others by analysing their formal definitions and conducting additional evaluations. 1. INTRODUCTION In January 2009, Arbitron and Edison Research measured the popularity of digital music platforms by means of a survey of 1,858 American people aged 12+ . 1 They esti- mated that 42 million Americans tune to online radio on a weekly basis, which is more than twice their number from 2005, and claim that the number of 12+ year old Ameri- cans owning a digital music player increased from 14% in 2005 to 42% in 2009. They also found that the vast ma- jority of these people own an Apple iPod or iPhone. Ev- idently, the Apple products dominate their market, which is commonly attributed to their innovating design and user interfaces. The recent “Genius” feature is a nice example of such innovation. Using this feature, users can automat- ically create coherent playlists by selecting a seed song, i.e., an example of a song of interest, and pressing a sin- gle button. Many of the popular online radio stations are similar in concept. The user supplies one or more seeds, and the system generates a corresponding list of tracks that is turned into a custom radio station. Hence, automatic playlist generation can be seen as a technology that is, to some extent, responsible for the recent growth established by certain digital music platforms, and its commercial im- portance is likely to increase further in the near future. This paper is about simple heuristics for automatically generating playlists. More precisely, we will discuss sim- 1 http://www.arbitron.com/study/digital_radio_ study.asp Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. c  2009 International Society for Music Information Retrieval. ple rules of thumb for choosing the song to be played next, given a set of candidate songs. This set of candidates can consist of all available tracks, but usually it is restricted to a limited subset. In order to avoid repetition, for instance, the set of candidates has to be restricted to the songs that have not been played yet. A realistic scenario is to select the candidates using some other method, effectively turn- ing the heuristic into an enhancement rather than a playlist generation method on its own. A very simple way to improve upon random selection, is to repeatedly choose the candidate that is most similar to a given seed song [1]. This playlist generation heuristic is said to be static because the song sequence is completely determined from the seed, without taking any additional user input into account. Dynamic heuristics, on the other hand, rely on user feedback to dynamically improve the se- lection process [2]. For example, the aforementioned static heuristic can be made dynamic by letting it pick the song that is most similar to any of the accepted songs, where the set of accepted songs consists of the seed song as well as all tracks that were not skipped [3]. When there is no given seed, the set of accepted songs can initially be empty and the next track can be chosen at random until there is at least one accepted song. This latter heuristic could eas- ily be added to any system that returns multiple candidate songs for being played next. Putting it in one sentence, we discuss simple dynamic playlist generation heuristics in this paper. In comparison with alternative techniques, such heuristics are interesting because they (i) are simple and thus easy to compute and implement, and (ii) can easily be added as an enhancement to many existing playlist generation systems. 2. RELATED WORK Dynamic playlist generation can be seen as a special case of the well-known relevance feedback paradigm from in- formation retrieval [4]. In this paradigm, the user is asked to give explicit feedback by labeling results as either rel- evant or irrelevant, which leads to additional information that can be used by the system to refine the search strat- egy and generate a better list of results. Several rounds of feedback can be conducted, each bringing the results closer to the user’s implicit target concept. Hence, dy- namic playlist generation is basically relevance feedback with the returned set of results restricted to one item. In case of this paper, the feedback taken into account is also implicit rather than explicit, but there is no reason to as- 351