Towards an Optimal Color Representation for Multiband Nightvision Systems Alexander Toet TNO Human Factors Soesterberg, The Netherlands. lex.toet@tno.nl Maarten A. Hogervorst TNO Human Factors Soesterberg, The Netherlands. maarten.hogervorst@tno.nl Abstract - We present a new Tri-band Color Low-light Observation (TRICLOBS) system The TRICLOBS is an all-day all-weather surveillance and navigation tool. Its sensor suite consists of two digital image intensifiers and an uncooled longwave infrared microbolometer. This sensor suite registers the visual, near-infrared and longwave infrared bands of the electromagnetic spectrum. The optical axes of the three cameras are aligned, using two dichroic beam splitters. A fast lookup- table based color transform (the Color-the-Night color mapping principle) is used to represent the TRICLOBS image in natural daylight colors (using information in the visual and NIR bands) and to maximize the detectability of thermal targets (using the LWIR signal). A bottom-up statistical visual saliency mode is deployed in the initial optimization of the color mapping for surveillance and navigation purposes. Extensive observer experiments will result in further optimization of the color representation for a range of different tasks. Keywords: Image fusion, false color, natural color mapping, real-time fusion, lookup tables, visual saliency. 1 Introduction Night vision cameras are a vital source of information for a wide-range of critical military and law enforcement applications related to surveillance, reconnaissance, intelligence gathering, and security. The two most common night-time imaging systems cameras are low- light-level (e.g., image-intensified) cameras, which amplify the reflected visible to near infrared (VNIR) light, and thermal infrared (IR) cameras, which convert invisible thermal energy from the midwave (3 to 5 microns) or the long wave (8 to 12 microns) part of the spectrum into a visible image. Until recently a gray- or greenscale representation of nightvision imagery has been the standard. However, the increasing availability of fused and multi-band infrared and visual nightvision systems has led to a growing interest in the color display of night vision imagery [9, 11, 12, 16, 23]. In principle, color imagery has several benefits over monochrome imagery for surveillance, reconnaissance, and security applications. For instance, color may improve feature contrast, which allows for better scene recognition and object detection. When sensors operate outside the visible waveband, artificial color mappings generally produce false color images whose chromatic characteristics do not correspond in any intuitive or obvious way to those of a scene viewed under natural photopic illumination. This type of false color imagery may disrupt the recognition process, resulting in an observer performance that is even worse compared to that obtained with singleband imagery alone [13]. Several different techniques have been proposed to display night-time imagery in natural daylight colors [14-17, 20, 23], some of which have been implemented in realtime nightvision systems [2, 8, 18, 19, 21]. Most of these techniques are computationally expensive and/or do not achieve color constancy. We recently introduced a new color mapping that displays night-time imagery in natural daytime colors [7]. This technique is simple and fast, and can easily be deployed in realtime. Moreover, it provides stable colorization under variations in scene content [6, 7]. Here we describe the implementation of this new color mapping in the prototype TRICLOBS (TRI-band Color Low-light OBServation) all-day all-weather surveillance and navigation system. The system displays the co- aligned visual, near-infrared and thermal signals of respectively two image intensifiers and an uncooled microbolometer in full color. A fast lookup-table implementation of the Color-the-Night color mapping transform2 is deployed to represent the TRICLOBS image in natural daylight colors (using information in the visual and NIR bands) and to maximize the detectability of thermal targets (using the LWIR signal). A bottom-up statistical visual saliency model [22] is deployed to optimize the color mapping for surveillance and navigation purposes. 12th International Conference on Information Fusion Seattle, WA, USA, July 6-9, 2009 978-0-9824438-0-4 ©2009 ISIF 1417