ROBUST NON-LINEAR FILTERING FOR VIDEO PROCESSING V. Zlokolica, W. Philips, D. Van De Ville Ghent University, Dept. of Telecommunications and Information Processing (TELIN), IPI Sint-Pietersnieustraat 41, 9000 Gent, Belgium Vladimir.Zlokolica@telin.rug.ac.be ABSTRACT Noise removal techniques such as the -nearest neighbour filter and the -trimmed mean filter are known to be very robust in still image noise removal, but they have not been exploited in video processing. In this paper we investigate their 3D-extension for use in video sequence noise removal. We also determine the optimal balance between temporal and spatial window size, the optimal values of the other parameters and finally we investigate the arte- facts introduced by the filters. The results show that the new video -nearest neighbour filter outperforms the video version of the - trimmed mean and the state-of-the-art rational filter by Ramponi from both a PSNR and a visual quality point of view. 1. INTRODUCTION Video sequences are often corrupted by noise, e.g., due to bad re- ception of television pictures. In the recent past a number of non linear techniques for video processing have been proposed, which are known as superior to linear techniques. However, most of the the existing non linear techniques work well only on specific types of noise, or have specific optimal parameter values for different types of the noise. For instance the state-of-the-art 3D rational filter [1–3] re- quires different parameters for, e.g., impulse and gaussian noise. Another important problem with most existing techniques is that they blur the sequence to some degree. In this paper, we develop techniques that are less sensitive to the noise type (e.g., impulse noise, gaussian noise or a mixture of both). In still image processing, two types of filters have been shown to be very robust w.r.t. noise type: the -trimmed mean filter [4] and the -nearest neighbour filter [5]. In this paper we extend these techniques to video processing by taking into account multiple frames. We investigate their performances and compare them to a state-of-the-art technique. We also present the video -trimmed mean filter which is an 3D extension of 1D -trimmed mean filter [4] later extended for image processing. The video -trimmed mean filter performs both spacial and temporal filtering of image sequence by sorting pixels within a 3D window in ascending order and averaging a certain number of pixels in the window, depending of the optimal para- meter value. However, it has been noticed that although it works well with impulsive noise, it doesn’t perform so good in case of gaussian noise. Finally, we introduce and stress the new video K-NN (K-nearest neigbour) filter which is an extension of a technique proposed earlier for image processing [6], and was first described by Davis and Rosenfield [5]. This is a non linear filter which sorts pixels within a 3D window, according to their difference with the central pixel’s value; after that, it averages the pixels in the window, after weighting them according to their sorting order. The results in this paper show that image sequences, corrup- ted with different types of noise have been well restored using the video K-NN filter without blurring the images. The main advant- age of the technique is that it works well on sequences independ- ently of the noise type (gaussian and impulse, or a combination of both), provided the impulse noise is not very big (i.e. not bigger then 10 ). In section 2 we introduce the new video filters and explain how they work. In section 3 we investigate the the optimal parameters in function of the noise level and noise type and we compare the performances of the filters to the 3D rational filter. The results show that the new video K-NN filter is superior to both the video -trimmed mean and the 3D rational filter. 2. NEW IMAGE SEQUENCE FILTERS In the following we denote an image pixel as , where and indicate the spatial and temporal location, respectively. We consider a window around . This window consists of the pixels where , and . This window is used to compute the filtered value . The whole sequence is filtered by sliding the 3D window over the pixels of the sequence. The 3D window is shown in fig. 1. In a first step, all the pixel values from the 3D window are put in a one dimensional array (in any particular order): (1) where denotes the number of the pixels in the 3D sliding window. 2.1. The video -trimmed mean filter After the pixel-extraction step (explained above), the second step for the video -trimmed filter is as follows: We sort the pixel values in in ascending order to get a new 1D array (2) where . The output of the filter is then calculated as follows: (3)