528 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 18, NO. 4, APRIL 2008 Parameter Embedding Mode and Optimal Post-Process Filtering for Improved WDCT Image Compression O˘ guzhan Urhan, Member, IEEE, and Sarp Ertürk, Member, IEEE Abstract—The warped discrete cosine transform (WDCT) has been shown to improve the compression performance compared to standard DCT for image compression at high bit rates. In the proposed approach, the WDCT parameters are embedded into the transform coefficients for low bit rates in the form of a watermark to avoid the WDCT parameter side information over- head, improving the compression performance for low bit rates. Furthermore, optimal postprocess filtering, with filter coefficients being determined at the encoder and multiplexed into the bit stream, is proposed to improve the quality of the decoded image through postprocess filtering at the decoder. It is shown that a significant gain in quality can be achieved by the postprocess filtering. As optimal postprocess filtering is utilized, the need of a deblocking filter for low bit rates is finally evaluated to improve the statistical quality and visual appearance. Index Terms—Embedded warping parameter, optimal filtering, warped discrete cosine transform (WDCT). I. INTRODUCTION B LOCK-BASED discrete cosine transform (DCT) encoders are used in many image and video coding standards as a re- sult of their high decorrelation performance and the availability of fast DCT algorithms enabling real-time implementation [1]. While DCT block coding provides good reproduction without noticeable artifacts at high or moderate bit rates, at low bit rates reconstructed images usually suffer from visual artifacts [2]. Different approaches have been proposed to improve the per- formance of DCT block encoders. One class of techniques proposed to improve the performance comprises methods that change or improve the transform. An adaptive discrete cosine transform that adjusts the transform ac- cording to input signal properties referred to as warped DCT (WDCT) is proposed in [3]. An important advantage of WDCT compared to the different transform approaches proposed to im- prove the performance of block coding, such as the interleaved block transform [4], the combined transform [5], and the lapped orthogonal transform [6], is that the WDCT can be implemented using conventional DCT software or hardware. Another class of techniques proposed to improve the perfor- mance of DCT block encoders comprises techniques that im- prove the quantization procedure to improve the performance of DCT block coding. It is shown that the performance can be improved compared to baseline JPEG by optimal quantization matrix design [7], coefficient thresholding [8], or joint coeffi- cient thresholding and quantizer design [9]. Manuscript received September 25, 2006; revised May 22, 2007. This paper was recommended by Associate Editor W. Gao. The authors are with the Electronics and Telecom Eng. Department, Ko- caeli University Laboratory of Image and Signal processing (KULIS), Univer- sity of Kocaeli, 41040 Turkey (e-mail: urhano@kou.edu.tr; sertur@kou.edu.tr; sarp@ieee.org). Digital Object Identifier 10.1109/TCSVT.2008.918769 Prefiltering and postfiltering are important tools to reduce compression artifacts in image compression schemes. The method proposed in [10] for example applies directional 1-D filtering for edge regions and uses 2-D adaptive average fil- tering for flat areas in order to reduce the blocking effect while keeping edges. Based on the same concept, the deblocking method proposed in [11] basically classifies blocky image re- gions as smooth, intermediate and complex regions and carries out suitable filtering according to the complexity of the region. Maximum a posteriori (MAP) [12] and projection onto convex sets (POCS) [13] are iterative procedures utilized to reduce compression artifacts. II. IMPROVED WDCT IMAGE CODING A. Warped Discrete Cosine Transform The 8-point DCT of an input vector can be defined as otherwise. (1) It is possible to carry out the DCT computation using a filter bank, where each filter is defined by (2) so that the th coefficient of is the th element of the DCT matrix [3]. The compression performance of DCT is known to suffer particularly for inputs with high-frequency components. It has been proposed in [3] to warp the input to ad- just its frequency distribution to be more suitable for DCT. A first order all-pass filter with transfer function (3) is used to perform the warping by replacing in (2) with . The frequency warping is controlled using the param- eter, and therefore this parameter is sent as side information for each block in [3]. In the utilized approach, approximated WDCT matrices are prepared using a set of warping parameters with values , , similar to [3]. The WDCT matrix for is equal to the conventional DCT matrix, hence the conventional DCT is actually a subset of WDCT. For each image block, every WDCT matrix is tried and the one that gives the lowest reconstruction error is selected. The index of the WDCT matrix (i.e., the index corresponding to the best value of 1051-8215/$25.00 © 2008 IEEE