Pre- and Post-Processing Algorithms for Compressed Video Enhancement C. Andrew Segall and Aggelos K. Katsaggelos {asegall, aggk}@ece.northwestern.edu Department of Electrical and Computer Engineering 2145 N. Sheridan Road Evanston, Illinois 60208-3118, USA. Abstract Standards-based video compression algorithms are rapidly becoming the preferred method for transmitting image sequences. Prominent examples include the MPEG and ITU family of standards. However, it is important to realize that these standards are not bit-exact, in that only the operation of the decoder is defined by the specification. Development of the rate-control mechanism and pre- and post-processing procedures is completely controlled by the system designer, and these components can introduce discernible differences between two standards compliant realizations. In this paper, we survey the fields of pre- and post-processing techniques for video compression. We then discuss our current work on compression enhancement algorithms. These algorithms are applicable to any compression standard but are discussed within the context of MPEG-2. 1. Introduction Digital video compression algorithms are a fundamental component of a video communication system. The goal of these algorithms is to represent a video sequence with a reduced number of bits, as compared to simply utilizing the intensity values of each pixel in the sequence. If the number of bits is reduced sufficiently, then transmitting and storing the video sequence becomes feasible with modern communication channels. These medias include Internet and Intranet delivery, wireless and satellite communications systems as well as DVD or hard disk storage devices. Compressed video sequences are usually imperfect descriptions of the original imagery, in that some of the visual data of the original sequence is discarded while encoding. Eliminating information reduces the number of bits required for the compressed representation. Unfortunately, it can also lead to visible errors in the decoded imagery. In this context, the overall visual quality of the decoded sequence then becomes a critical factor in the design and comparison of video compression systems. Many variables affect this quality. For example, rate-control algorithms allocate available bits to different parts of an image sequence, which can introduce a visible difference between similar compression algorithms. In the same way, the design and implementation of pre- and post-processing mechanisms often differentiates one product from another. In this paper, we survey the field of pre- and post- processing algorithms for digital video compression and then discuss our current work in the area. For both types of algorithms, the goal of the operation is to reduce the severity of coding artifacts within the reconstructed video sequence. Pre-processing algorithms realize this goal by filtering the original images before (or during) compression. This improves performance by simplifying the image content that must be encoded. Post-processing algorithms operate after a sequence is compressed and do not have knowledge of the original image data. Instead, these algorithms identify and attenuate coding artifacts that were introduced by the encoder. 2. Background Pre- and post-processing algorithms are usually designed within the framework of a successful video standard. At the current time, these standards include the ITU and MPEG compression mechanisms. In these methods, each frame of a video sequence is coded with a combination of a block-wise transformation with the Discrete Cosine Transform (DCT) and motion compensation. Compression is realized by quantizing the transform coefficients. Unfortunately, this quantization can introduce visible errors in the reconstructed video sequence. These artifacts are typically classified as blocking, ringing or temporal flicker. The independent processing of each block results in the first class of coding artifacts. These blocking errors dominate low bit-rate applications, where the encoder typically removes all high-frequency information. The result is a series of piece-wise constant image blocks, with sharp transitions at the block boundaries. Increasing the bit-rate of the compression system helps in reducing the blocking artifacts. However, higher bit-rate applications are still plagued by the visible "grid-like" structure in the decompressed data. This work was supported in part by a grant from the Motorola Center for Communications.