A Robust Preprocessing Algorithm for Low-Resolution Soccer Videos Martin F. Wrulich, Luca Superiori, Olivia Nemethova and Markus Rupp Vienna University of Technology Institute of Communications and RF Engineering Gusshausstraße 25/389,1040 Vienna, Austria mwrulich@nt.tuwien.ac.at, lsuper@nt.tuwien.ac.at, onemeth@nt.tuwien.ac.at, mrupp@nt.tuwien.ac.at ABSTRACT Multimedia transmissions, especially of sport events, are an important share of the services offered by mobile network op- erators. Due to limited capacity, lossy compression codecs have to be used, inducing a significant quality degradation. The quality impairments can particularly be observed in soc- cer games, when the ball is blurred or even totally disappears on low-resolution user terminals. We present a suitable pre- processing of the original video by means of ball sharpening or enlarging in the original video sequence to avoid problems caused by resolution down-sampling and compression. The proposed system includes a novel initial ball-recognition al- gorithm and an improved ball-trajectory tracking. Our pro- posed algorithm runs robustly and is computational very economical. Categories and Subject Descriptors I.4 [Image Processing and Computer Vision]: Appli- cations General Terms Algorithms, Experimentation Keywords Visual Content Analysis, Image Processing, Ball Tracking, Recognition, Segmentation 1. INTRODUCTION Nowadays, soccer games represent very popular contents not only for analog and digital television, but also for stream- ing over mobile networks. Typical mobile terminals usually offer resolutions as small as 144×176 (QCIF). The Universal Mobile Telecommunication System (UMTS) 1 supports data 1 European standard of the 3rd generation of mobile sys- Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. MV’07, September 28, 2007, Augsburg, Bavaria, Germany. Copyright 2007 ACM 978-1-59593-779-7/07/0009 ...$5.00. rates up to 2 Mbit/s, shared by all users in a cell. There- fore, for unicast transmission of streaming video, data rates about up to 128 kbit/s are feasible. Video codecs supported by UMTS are currently H.263 and MPEG-4, with their basic profiles, H.264 is in discussion (see [1]). Lossy compression used by these codecs leads to visual quality degradation: frame reduction causes an overall jerkiness of the video; fur- ther compression achieved by the coarser quantization re- sults in loss of spatial details accompanied with blockiness and blurriness. A soccer match usually encompasses scenes with diverse character, separated by scene changes (i.e. cut, wipe, zoom in/out, transition, etc.). Most common are wide-angle mov- ing camera shots. These are particularly critical for the com- pression, since the ball as well as the players are represented by a few pixels only, and thus very susceptible to any qual- ity degradation. The ball in a soccer game carries the most important information; watching a game with a small ball blurred into the field becomes rather annoying. In some cases the ball even disappears from the image due to the compression or due to picture resolution down-sampling. Figure 1 provides a schematic illustration of the problem: the ball in the already downsampled input sequence is rep- resented by several pixels of not necessarily uniform color. The appearance of the ball differs even within the same video sequence from frame to frame. After video compression (per- formed before the transmission) the ball visibility worsens considerably. To improve the ball appearance in such cases, we already proposed a method [2] that searches the ball in each frame and replaces it by its enlarged or just sharpened version. The so replaced ball is then likely to be visible af- ter the compression and promises to considerably enhance the user perceptual quality. The main advantage of such a solution is its wide applicability: it does not require any changes of the video codec or network protocol standards and the method can run at the streaming server of the mo- bile operator or the content provider. In this paper, complementary to the findings in [2], we present a new initial ball-recognition algorithm, which is able to reliably detect the ball at the beginning of a scene without supervision. Furthermore, we introduce an MMSE (minimum mean square error) based ball-tracking to predict a suitable region of interest for the on-the-fly ball detection together with a simple but effective template update. Our proposed techniques are focused on affordable complexity, tems, standardized by 3rd Generation Partnership Project (3GPP) 61 ACM Workshop Mobile Video 2007, Augsburg, Germany, Sep. 2007.