A NEW DATA HIDING SCHEME FOR SCENE CHANGE DETECTION IN H.264 ENCODED VIDEO SEQUENCES Spyridon K. Kapotas and Athanassios N. Skodras Digital Systems and Media Computing Laboratory, School of Science and Technology Hellenic Open University Patras, Greece {s.kapotas, skodras}@eap.gr ABSTRACT A new data hiding scheme for scene change detection in H.264 streams is presented. The proposed scheme takes advantage of the different block sizes used by the H.264 encoder during the inter prediction stage in order to hide the scene change information inside the encoded sequence. This information enables real time scene change detection directly from the H.264 stream. Keywords—H.264, data hiding, scene change detection, video indexing, video retrieval 1. INTRODUCTION Scene change detection is an important means for video editing, video indexing, error resilience, etc. and it has been recognized as one of the significant research areas in recent years. Shot is as a sequence of frames captured by a single camera in a single continuous action in time and space. A video scene or story unit is defined as a group of content correlated (in time, space, etc.) shots, which is a higher level semantic unit. The widespread of the Internet and World Wide Web has changed the way digital data are handled. As the H.264 standard is becoming the dominant standard in video coding, the easy access of the H.264 encoded video sequences requires efficient video indexing and archiving. Moreover, due to the nature of the H.264 standard (high compression, low bitrate, etc.), most of the H.264 encoded sequences are available through the Web. There is, therefore, a great need for fast and efficient real time scene change detection methods, which can be applied directly onto H.264 bitstreams. Many scene change detection methods can be found in the literature, but some of them are either computationally expensive or ineffective. In the uncompressed domain, the major techniques are based on pixels, histogram comparisons and/or edge difference calculations [1-3]. Recent trends focus on developing scene change detection algorithms directly in the compressed domain, especially for MPEG compressed videos [4-7]. In general, the methods, which work in the uncompressed domain are more efficient but not as useful as the methods that work in the compressed domain. In this paper we propose a new data hiding scheme, which can be combined with any existing uncompressed domain scene change detection method in order to enable real time scene change detection in the compressed domain. The idea is to detect the scene change during the H.264 encoding process and hide this information as metadata in the encoded sequence. It is then easy for a metadata aware application to detect the scene change, and possibly extract other useful information about the scene, in the compressed domain. The scheme is embedded in the H.264 encoder and it takes advantage of the different block sizes used by the encoder during the inter prediction, in order to hide the meta data. The data can then be extracted directly from the encoded stream without knowing the original host video. The idea of using metadata hiding in video, audio and images in order to create data channels is not new. Metadata hiding has also been used for error correction [8] and for content adaptation [9]. The rest of the paper is organized as follows. In Section 2 we present the H.264 inter prediction process, which is an essential part of our scheme. In Section 3 we describe the new scheme. In Section 4 the metadata extractor is briefly described. In Section 5 we present the simulation results. In Section 6 some further improvements are discussed. Finally, in Section 7 conclusions are drawn. 2. THE H.264 INTER PREDICTION SCHEME As specified in H.264/AVC standard [10], there are 7 different block sizes (16x16, 16x8, 8x16, 8x8, 8x4, 4x8 and 4x4) that can be used in inter frame motion estimation/ compensation. These different block sizes actually form a two-level hierarchy inside a macroblock. The first level comprises block sizes of 16x16, 16x8 or 8x16. In the second level the macroblock is specified as P8x8 type, of which each 8x8 block can be one of the subtypes such as 8x8, 8x4, 4x8 or This work was funded by the European Union - European Social Fund (75%), the Greek Government - Ministry of Development - General Secretariat of Research and Technology (25%) and the Private Sector in the frames of the European Competitiveness Programme (Third Community Support Framework - Measure 8.3 programme PENED - contract no. 03ED832).