BACKWARD COMPATIBLE ENHANCEMENT OF CHROMA FORMAT IN HEVC D. Bugdayci Sansli ⋆ K. Ugur † M.M Hannuksela † M. Gabbouj ⋆ ⋆ Tampere University of Technology, Tampere, Finland † Nokia Research Center, Tampere, Finland ABSTRACT First version of the latest video coding standard, High Effi- ciency Video Coding (HEVC), only supports coding of video in YUV 4:2:0 chroma format. An extension of the standard that will support other chroma formats is currently under de- velopment, however, version 1 decoders will not be able to handle the bitstreams created using this extension. In this pa- per, we propose a novel method to create scalable bitstreams that involve a backward compatible base layer in 4:2:0 for- mat that can be handled by HEVC version 1 decoders and code additional layers to enhance the chroma resolution. The proposal codes 4:2:0 video in the base layer and the high resolution chroma components as auxiliary pictures as sepa- rate enhancement layers. The high resolution chroma compo- nents could optionally be predicted from the upsampled 4:2:0 chroma components of base layer. The simulations show that the proposed method achieves scalability with 9.5% coding efficiency penalty on average compared to single layer cod- ing of 4:4:4 video. When compared to simulcast of 4:2:0 and 4:4:4 video, proposed method provides 38% gain on average. Proposed method makes services using high chroma fidelity easier to be deployed, due to the backwards compatibility to existing HEVC implementations with high coding efficiency. Index Terms— HEVC, chroma format, subsampling ra- tio, scalability, backward compatibility 1. INTRODUCTION In video compression schemes, chroma subsampling ratios are determined according to the requirements of the appli- cation. For broadcasting, web or storage (DVDs, hard disks etc.), most commonly used ratio is 4:2:0 which is derived by an RGB-to-YCbCr color-space transformation and subsam- pling of the chroma components by a factor of 2:1 both hor- izontally and vertically. However, professional video codecs such as those found in HD cameras etc. prefer to use higher colour resolution as in 4:4:4 where all channels have the same resolution or 4:2:2 where chroma components are subsampled by a factor of 2:1 horizontally only. There are also applica- tions emerging where the usage of higher chroma fidelity is preferred. These applications involve mostly coding of com- puter generated content such as screen sharing in a video con- ferencing session, remote desktop applications or streaming the user interface of a mobile device to a TV (such as wireless display - WiDi), where subsampling of chroma introduces vi- sually annoying artifacts. First version of the new video compression standard, HEVC, is published in early 2013. It is jointly developed by the ISO/IEC Moving Picture Experts Group (MPEG) and ITU-T Video Coding Experts Group (VCEG) and it supports only YUV 4:2:0 chroma format [1]. Development of range extensions which addresses coding of chroma formats other than 4:2:0 and higher bit depths is currently ongoing. The new extension is designed to have additional low level tools for improved coding of different chroma formats and higher bit-depths. From the network point-of-view, it is desirable to support chroma formats other than 4:2:0 in a backward compatible way. In this paper, we propose methods to create layered bitstreams which include a backward compatible base layer for HEVC version 1 decoders and chroma enchance- ment layers where only high-level changes, such as slice header decoding or above, to 4:2:0 decoding operation would be needed to decode the chroma enhancement. Another way to provide backward compatible 4:4:4 con- tent is to have chroma format scalability where base layer and enhancement layers have different chroma formats. However, currently this is not possible for HEVC, as scalable exten- sion supports 4:2:0 only. During the development of SVC, chroma format scalability was studied but it was decided not to be included in the standard as it does not provide sufficient gains [2]. The method described in [3] provides a way to con- vey high resolution colour content (specifically 4:4:4 chroma format) through a 4:2:0 decoder. Unlike our proposal, this method uses an unmodified codec but it requires pre-encoding and post-decoding operations. The main idea is to create two 4:2:0 frames for each 4:4:4 frame and combine them into one frame with twice the height as in top-bottom frame packing of stereo content. Top part of the new frame consists of the sub- sampled version of 4:4:4 content (full resolution luma and quarter resolution chroma) and bottom part consists of the re- maining chroma. Frame packing arrangement SEI message is used to inform the decoder. In Section 2, the proposal is described in details and var- ious ways of implementing it are discussed. In Section 3 test conditions and simulations are explained, experimental