DATA PRUNING-BASED COMPRESSION USING HIGH ORDER EDGE-DIRECTED INTERPOLATION D˜ ung Trung V˜ o 1 * , Joel Sol´ e 2 , Peng Yin 2 , Cristina Gomila 2 , Truong Nguyen 1 1 Video Processing Laboratory, UC San Diego. 9300 Gilman Drive, La Jolla, CA 92092. 2 Thomson Corporate Research. 2 Independence Way, Princeton, NJ 08540. ABSTRACT This paper proposes a data pruning-based compression scheme to improve the rate-distortion relation of compressed images and video sequences. The original frames are pruned to a smaller size before compression. After decoding, they are interpolated to their original size by an edge-directed interpolation. The data pruning is optimized to obtain the minimal distortion in the interpolation phase. Further- more, a novel high order interpolation is proposed to adapt the inter- polation to many edge directions. This high order ſltering uses extra surrounding pixels and achieves more robust edge-directed image interpolation. Simulation results are shown for both image interpo- lation and coding applications. Index Terms— interpolation, data pruning, compression, spa- tial ſltering, edge-directed interpolation, video coding. 1. INTRODUCTION Nowadays, the request for higher quality video is emerging very fast. Video tends to higher resolution, higher frame-rate and higher bit-depth. New technologies to further reduce bit-rate are strongly demanded to combat the bit-rate increase of this high deſnition video, especially to meet the network and communication transmis- sion constraints. In video coding, there are two main directions to reduce compression bit-rate. One is to improve the compression technology and the other one is to perform a preprocessing before compression. The ſrst direction can be seen from the development of the MPEG video coding standard, from MPEG-1 to H.264/MPEG-4 AVC. For most video coding standards, increasing quantization step size is used to reduce bit-rate [1]. However, this technique can result in blocky artifacts and other coding artifacts due to the loss of high frequency details. In the second direction, common techniques are low-pass ſltering or downsampling (which can be seen as a ſltering process) followed by reconstructing or upsampling at the decoder. For example, low-pass ſlters were adaptively used based on Hu- man Visual System to eliminate high frequency information in [2] or to simplify the contextual information in [3]. Also, to reduce the bit-rate, some digital television systems uniformly downsized the original sequence and upsized it after decoding. The recon- structed video applying these techniques looked blur because they were designed to eliminate high-frequency information with the anti-aliasing ſlter before downsizing or with the low-pass ſlter in the preprocessing step. ∗ This work is done while D˜ ung Trung V˜ o was with Thomson Corporate Research. This paper proposes a novel data pruning-based compression scheme to reduce the bit-rate while still keeping a high quality re- constructed frame. The original frames are ſrst optimally pruned to a smaller size by adaptively dropping rows or columns prior to en- coding. At the ſnal stage, an interpolation phase is implemented to reconstruct the decoded frames to their original size. By avoiding ſltering the remaining rows and columns, the reconstructed frames can achieve high quality from a lower bit-rate. Main applications of interpolation are upsampling, demosaick- ing and displaying in different video formats. A wide range of in- terpolation methods has been discussed, starting from conventional bilinear and bicubic interpolations to sophisticated iterative interpo- lations such as projection onto convex sets (POCS) [4] and noncon- vex nonlinear partial differential equations [5]. Another group of in- terpolation algorithms predicted the ſne structure of the high resolu- tion (HR) image from its low resolution (LR) version using different kinds of transform such as wavelet [6] or contourlet transform [7]. To avoid the jerkiness artifacts occurring along edges, edge-oriented interpolation methods were performed using Markov random ſeld [8] or the LR image covariance [9]. All the above methods are for upsampling the same ratio in both horizontal and vertical directions. However, when interpolation is used along with data pruning, the method needs to adapt to the way of pruning the data and to the structure of surrounding pixels. For instance, there are pruning cases in which only rows or only columns are dropped and upsampling in only one direction is required. This paper develops a high order edge-directed interpolation scheme to deal with these cases. Another algorithm is also considered for the cases of dropping both rows and columns. The paper is organized as follows. Section II introduces the data pruning-based compression method and derives an optimal data pruning algorithm. Section III describes the high order edge-directed interpolation method. Results for interpolation and coding applications are presented in Section IV. Finally, Section V gives the concluding remarks. 2. OPTIMAL DATA PRUNING The block diagram of the data pruning-based compression is shown in Fig. 1. Assume that the original frame I of size M ×N is pruned to frame P of smaller size (M - Mp) × (N - Np), where Mp and Np are the number of dropped rows and columns, respectively. The purpose of data pruning is to reduce the number of bits representing the stored or compressed frame P ′ . Then, I ′ of the original size is obtained by interpolating P ′ . In this paper, only the even rows and columns may be discarded, while the odd rows and columns are always kept for later interpola- tion. The block diagram of the data pruning phase is shown in Fig. 2. 997 978-1-4244-2354-5/09/$25.00 ©2009 IEEE ICASSP 2009