A HIGH THROUGHPUT ENCODER FOR HIGH DYNAMIC RANGE IMAGES Firas Hassan and Joan Carletta Department of Electrical and Computer Engineering The University of Akron Akron, Ohio USA ABSTRACT An encoder for high dynamic range (HDR) images is proposed that is compatible with the JPEG2000 compression engine and can provide three different versions of the image to the user at the receiver end depending on his needs. The first version is a gray scale HDR image. The second is a local tone-mapped version of the color HDR image suitable for display on conventional devices. By combining data from the first two versions, the original color HDR image can be reconstructed. The proposed HDR encoder was implemented on a field programmable gate array, and has high enough throughput to support a system processing 1024×768 images at a rate of 60 frames per second. Index Terms— high dynamic range, field programmable gate array, JPEG2000 1. INTRODUCTION The sheer volume of data required to represent a HDR image is a bottleneck limiting the use of HDR images. Current research is investigating encoding of HDR images, removing redundant or perceptually unimportant information, in ways suitable to interface with lossy compression engines such as JPEG2000. It is highly desirable to be able to do this encoding in real-time, and embedded within a camera. However, published HDR encoders have been implemented mostly on workstations and graphics cards without regard to resource or time constraints. The work described in this paper develops a HDR image encoder that operates in real-time on an embedded system with high throughput and low cost. The paper is organized as follows. Section 2 describes related work, while Section 3 describes the proposed HDR image encoder in detail. Section 4 shows the results obtained from synthesizing and verifying the design. Section 5 draws conclusions. 2. BACKGROUND The pioneering compression algorithms for HDR images, such as RGBE [1] and LogLuv [2], used lossless methods and were not efficient. An early HDR image encoder compatible with lossy compression techniques was presented in [3]. Mantiuk et al. transform the HDR data to the Luv space (similar to logLuv) and quantize the color components linearly. Then they use a global tone mapping operator to perceptually quantize the luminance component. The resulting image is sent to an MPEG video compressor. In [4], Mantiuk et al. introduced a compact reconstruction function that is used to decompose the HDR image into a standard low dynamic range (LDR) stream and a residual stream that together can be used to reconstruct the original image. A similar approach was suggested by Ward et al. in [5]. Both [4] and [5] targeted MPEG video compression; because the MPEG protocol does not allow for additional channels, the residual image must be sent in a subband. As a result, the methods concentrated on decorrelating the residual from the LDR image and coding the data so that it will fit in a subband. Use of JPEG2000 for high dynamic range applications has two advantages: JPEG2000 allows for more precision than MPEG, and it allows for additional channels that can be used to send luminance data. Xu et al. develop in [6] an encoder for HDR images that can be used as a front end for a JPEG2000 compression engine. They send a 16-bit logarithm of every color component to the engine. However, they are not able to downsample the chrominance red and chrominance blue inside the compression engine (as is usually done in JPEG2000 to decrease the size of the compressed image) because in the log domain the chrominance spaces bear significant information. Finally, Munkberg et al. and Roimela et al. suggest textural compression of HDR images in [7] and [8], respectively; both methods were implemented on graphics cards. Our encoder uses a local tone mapping operator that is a variant of the operator suggested by Reinhard et al. in [9]. This operator has been implemented on graphics cards by Goodnight et al. in [10] and Krawczyk et al. in [11]. The implementation in [11] processes 1024×768 images at a rate of 14 frames per second; this is not considered real-time. Our work, which implements a hardware- friendly variant of the Reinhard operator inside the encoder, has been implemented on a field programmable gate array and has high enough throughput for real-time applications. 3. HDR IMAGE ENCODER The block diagram of our HDR image encoder is shown in Figure 1 and is made up of five subblocks. At the input, the high dynamic range image is represented as an RGB triplet, where each element is a 32-bit fixed-point value, with 16 bits of integer and 16 bits of fraction. The luminance calculation subblock scales the RGB triplet by 2 16 in order to transform the numbers to integer and VI - 213 1-4244-1437-7/07/$20.00 ©2007 IEEE ICIP 2007