EPG content recommendation in large scale: a case study on interactive TV platform D´ avid Zibriczky, Zolt´ an Petres, M´ arton Waszlavik, Domonkos Tikk Gravity R&D Zrt. 9025 Gy˝ or, Hungary E-mail: {david.zibriczky, zoltan.petres, marton.waszlavik, domonkos.tikk}@gravityrd.com Abstract—Recommender systems in TV applications mostly focusing on the recommendation of video-on-demand (VOD) con- tent, though the major part of users’ content consumption is real- ized on linear channel programs, termed EPG content. In this case study we present how we tackled the EPG recommendation task, which exhibits several differences compared to the VOD scenario, including the lack of explicit user feedbacks, the magnitude of cold start problem, as well as data cleaning and feature selection necessary to be applied on raw consumption data. We provide both offline and online model validation. First we showcase the typical approach in machine learning by evaluating models against recall in an offline setting. Then, we investigate in depth the real-world results of the recommendation app using the pre- trained models, and analyze how personalized recommendation influence users watching behavior. The experimentation results are based on our recommender system deployed at a Canadian IPTV service provider using Microsoft Mediaroom middleware. Keywords—EPG content recommendation, implicit feedback, collaborative filtering, matrix factorization, offline evaulation, on- line evaluation I. I NTRODUCTION Content consumption trends changed significantly in the last 10 years with the rise of digital evolution and Internet. Netflix and YouTube were the two main innovators in the last decade. Netflix streaming service allows reaching thousands of blockbuster and premium Hollywood contents, a selection that never before was accessible from a digital device. YouTube also became the de facto main content hub where all kinds of content can be found. To keep subscribers, TV operators expanded their media offerings by adding more live channels, enabled on-demand content catalogues, and added premium functions like PVR (personal video recordings) to allow time- shift content watching. Today TV services oftentimes offer over 500 live channels and provide video libraries with tens of thousands titles. However, as more and more channels and on-demand titles are added, the navigation becomes more difficult and challeng- ing, just like for its Internet counterparts Netflix and YouTube. For this, the quick overview provided by EPG (electronic program guide) is not sufficient any more, since it covers only 8–10 channels for 3–4 hours timespan on one screen, thus necessitating much of scrolling for navigation and selection. Service operators thus realized, that with over 500 of available live channels a completely different approach is needed, to help users finding relevant content in the plethora of program offerings, a recommendation based solution using advanced data mining and user profiling techniques like Netflix. In this paper, we focus on the less studied EPG content recommendation task, which has still much larger business impact than on-demand services. 1 EPG-recommendation ex- hibits several major differences compared to Netflix-like VOD- recommendation scenario that one should tackle [1]. • Noisy input: At TV watching users do not provide explicit feedback, the taste is expressed implicitly by channel zapping, recording, recording playback events. One key challenge is how to identify the relevant user actions that can be efficiently used both for user profiling and for the evaluation of recommendations. • Live TV programs are always new: A significant portion of EPG-programs are live, therefore always new to every- one. Hence, recommendation algorithms cannot use his- torical data. The metadata associated with live programs is also much less detailed and relevant than on-demand programs. About 60% of live TV programs belong to news, sports, reality, talk shows, and many of them are aired in live. The concise program descriptions available in EPG provides very limited information, making the recommendation challenging. • Time-based recommendation: Consumption patterns changes on daily and weekly bases, users are interested in different content in the morning, in the evening, or during the weekend. In [1], we proposed solutions to these tasks and performed an offline evaluation. After a brief summary of the obtained results, here we further investigate whether and how offline re- sults are translated into real-world measurements, by analyzing the influence of recommendations on user behavior. For this, we use the EPG-consumption data of the recommendation app deployed at Sasktel, a Canadian IPTV provider. We show in the analysis how users interact with the recom- mended content in terms of click-through rate. Also, we found that users of the recommendation app watch recommended content proportionally longer than other programs, indicating that the preference modeling efficiently helps user finding programs relevant for them. We also show that offline recall and mean reciprocal rank results are translated into online watching ratio, having the same tendency over different content types. 1 The generated income from subscription fees is still much larger than from the on-demand services, therefore solving the troublesome navigation issue is crucial. If subscribers do not see the value of the service offering, they just leave and use other services.